java.lang.Object
org.elasticsearch.snapshots.SnapshotsServiceUtils
A utility class for static snapshotting methods.
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic voidaddCloneEntry(List<SnapshotsInProgress.Entry> entries, SnapshotsInProgress.Entry entryToUpdate, ImmutableOpenMap.Builder<RepositoryShardId, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments) static voidaddSnapshotEntry(List<SnapshotsInProgress.Entry> entries, SnapshotsInProgress.Entry entryToUpdate, ImmutableOpenMap.Builder<ShardId, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments) static booleanAssert that there are no snapshots that have a shard that is waiting to be assigned even though the cluster state would allow for it to be assignedbuildGenerations(SnapshotsInProgress.Entry snapshot, Metadata metadata) static <T> voidcompleteListenersIgnoringException(List<ActionListener<T>> listeners, T result) static List<SnapshotsInProgress.Entry> currentSnapshots(SnapshotsInProgress snapshotsInProgress, ProjectId projectId, String repository, List<String> snapshots) Returns status of the currently running snapshotsstatic SnapshotDeletionsInProgressdeletionsWithoutSnapshots(SnapshotDeletionsInProgress deletions, Collection<SnapshotId> snapshotIds, ProjectId projectId, String repository) Remove the givenSnapshotIds for the givenrepositoryfrom an instance ofSnapshotDeletionsInProgress.static voidensureNoCleanupInProgress(ClusterState currentState, String repositoryName, String snapshotName, String reason) Checks the cluster state for any in-progress repository cleanup tasks (RepositoryCleanupInProgress).static voidensureNotReadOnly(ProjectMetadata projectMetadata, String repositoryName) static voidensureRepositoryExists(String repoName, ProjectMetadata projectMetadata) ThrowsRepositoryMissingExceptionif no repository by the given name is found in the given cluster state.static voidensureSnapshotNameAvailableInRepo(RepositoryData repositoryData, String snapshotName, Repository repository) static voidensureSnapshotNameNotRunning(SnapshotsInProgress runningSnapshots, ProjectId projectId, String repositoryName, String snapshotName) static <T> voidfailListenersIgnoringException(List<ActionListener<T>> listeners, Exception failure) static Map<String, DataStreamAlias> filterDataStreamAliases(Map<String, DataStream> dataStreams, Map<String, DataStreamAlias> dataStreamAliases) Filters out the aliases that refer to data streams to do not exist in the provided data streams.static booleanincludeFileInfoWriterUUID(IndexVersion repositoryMetaVersion) static booleanincludesUUIDs(IndexVersion repositoryMetaVersion) Checks whether the metadata version supports writing the cluster- and repository-uuid to the repository.initShardSnapshotStatus(ShardGeneration shardRepoGeneration, ShardRouting primary, Predicate<String> nodeIdRemovalPredicate) Compute the snapshot status for a given shard based on the current primary routing entry for the shard.static booleanstatic booleanChecks if the givenSnapshotsInProgress.Entryis currently writing to the repository.static voidlogSnapshotFailure(String operation, Snapshot snapshot, Exception e) maybeAddUpdatedAssignment(ImmutableOpenMap.Builder<T, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments, SnapshotsInProgress.ShardSnapshotStatus finishedShardState, T shardId, Map<T, SnapshotsInProgress.ShardSnapshotStatus> statesToUpdate) static IndexVersionminCompatibleVersion(IndexVersion minNodeVersion, RepositoryData repositoryData, Collection<SnapshotId> excluded) Determines the minimumIndexVersionthat the snapshot repository must be compatible with from the current nodes in the cluster and the contents of the repository.static List<SnapshotFeatureInfo> onlySuccessfulFeatureStates(SnapshotsInProgress.Entry entry, List<String> finalIndices) Removes all feature states which have missing or failed shards, as they are no longer safely restorable.processWaitingShardsAndRemovedNodes(SnapshotsInProgress.Entry snapshotEntry, RoutingTable routingTable, DiscoveryNodes nodes, Predicate<String> nodeIdRemovalPredicate, Map<RepositoryShardId, SnapshotsInProgress.ShardSnapshotStatus> knownFailures) Walks through the snapshot entries' shard snapshots and creates applies updates from looking at removed nodes or indexes and known failed shard snapshots on the same shard IDs.static ProjectMetadataprojectForSnapshot(SnapshotsInProgress.Entry snapshot, ProjectMetadata project) readyDeletions(ClusterState currentState, ProjectId projectId) Finds snapshot delete operations that are ready to execute in the givenClusterStateand computes a new cluster state that has all executable deletes marked as executing.static booleanremovedNodesCleanupNeeded(SnapshotsInProgress snapshotsInProgress, List<DiscoveryNode> removedNodes) shards(SnapshotsInProgress snapshotsInProgress, SnapshotDeletionsInProgress deletionsInProgress, ProjectState currentState, Collection<IndexId> indices, boolean useShardGenerations, RepositoryData repositoryData, String repoName) Calculates the assignment of shards to data nodes for a new snapshot based on the given cluster state and the indices that should be included in the snapshot.static org.apache.logging.log4j.LevelsnapshottingDataStreams(ProjectState projectState, Set<String> dataStreamsToCheck) Returns the data streams that are currently being snapshotted (with partial == false) and that are contained in the indices-to-check set.snapshottingIndices(ProjectState projectState, Set<Index> indicesToCheck) Returns the indices that are currently being snapshotted (with partial == false) and that are contained in the indices-to-check set.static ClusterStatestateWithoutSnapshot(ClusterState state, Snapshot snapshot, FinalizeSnapshotContext.UpdatedShardGenerations updatedShardGenerations) Computes the cluster state resulting from removing a given snapshot create operation from the given state.static booleansupportsNodeRemovalTracking(ClusterState clusterState) static ClusterStateupdateWithSnapshots(ClusterState state, SnapshotsInProgress snapshotsInProgress, SnapshotDeletionsInProgress snapshotDeletionsInProgress) Shortcut to build newClusterStatefrom the current state and updated values ofSnapshotsInProgressandSnapshotDeletionsInProgress.static booleanuseIndexGenerations(IndexVersion repositoryMetaVersion) Checks whether the metadata version supports writingShardGenerationsto the repository.static booleanuseShardGenerations(IndexVersion repositoryMetaVersion) Checks whether the metadata version supports writingShardGenerationsto the repository.static voidstatic voidvalidate(String repositoryName, String snapshotName, ProjectMetadata projectMetadata) Validates snapshot requeststatic booleanwaitingShardsStartedOrUnassigned(SnapshotsInProgress snapshotsInProgress, ClusterChangedEvent event)
-
Constructor Details
-
SnapshotsServiceUtils
public SnapshotsServiceUtils()
-
-
Method Details
-
ensureSnapshotNameNotRunning
public static void ensureSnapshotNameNotRunning(SnapshotsInProgress runningSnapshots, ProjectId projectId, String repositoryName, String snapshotName) -
ensureNoCleanupInProgress
public static void ensureNoCleanupInProgress(ClusterState currentState, String repositoryName, String snapshotName, String reason) Checks the cluster state for any in-progress repository cleanup tasks (RepositoryCleanupInProgress). Note that repository cleanup is intentionally cluster wide exclusive. -
ensureNotReadOnly
-
ensureSnapshotNameAvailableInRepo
public static void ensureSnapshotNameAvailableInRepo(RepositoryData repositoryData, String snapshotName, Repository repository) -
ensureRepositoryExists
ThrowsRepositoryMissingExceptionif no repository by the given name is found in the given cluster state. -
validate
public static void validate(String repositoryName, String snapshotName, ProjectMetadata projectMetadata) Validates snapshot request- Parameters:
repositoryName- repository namesnapshotName- snapshot nameprojectMetadata- current project metadata
-
validate
-
assertNoDanglingSnapshots
Assert that there are no snapshots that have a shard that is waiting to be assigned even though the cluster state would allow for it to be assigned -
useShardGenerations
Checks whether the metadata version supports writingShardGenerationsto the repository.- Parameters:
repositoryMetaVersion- version to check- Returns:
- true if version supports
ShardGenerations
-
useIndexGenerations
Checks whether the metadata version supports writingShardGenerationsto the repository.- Parameters:
repositoryMetaVersion- version to check- Returns:
- true if version supports
ShardGenerations
-
includesUUIDs
Checks whether the metadata version supports writing the cluster- and repository-uuid to the repository.- Parameters:
repositoryMetaVersion- version to check- Returns:
- true if version supports writing cluster- and repository-uuid to the repository
-
includeFileInfoWriterUUID
-
supportsNodeRemovalTracking
-
isWritingToRepository
Checks if the givenSnapshotsInProgress.Entryis currently writing to the repository.- Parameters:
entry- snapshot entry- Returns:
- true if entry is currently writing to the repository
-
isQueued
-
buildGenerations
public static FinalizeSnapshotContext.UpdatedShardGenerations buildGenerations(SnapshotsInProgress.Entry snapshot, Metadata metadata) -
projectForSnapshot
public static ProjectMetadata projectForSnapshot(SnapshotsInProgress.Entry snapshot, ProjectMetadata project) -
currentSnapshots
public static List<SnapshotsInProgress.Entry> currentSnapshots(@Nullable SnapshotsInProgress snapshotsInProgress, ProjectId projectId, String repository, List<String> snapshots) Returns status of the currently running snapshotsThis method is executed on master node
- Parameters:
snapshotsInProgress- snapshots in progress in the cluster stateprojectId- project to look for the repositoryrepository- repository idsnapshots- list of snapshots that will be used as a filter, empty list means no snapshots are filtered- Returns:
- list of metadata for currently running snapshots
-
processWaitingShardsAndRemovedNodes
public static ImmutableOpenMap<ShardId,SnapshotsInProgress.ShardSnapshotStatus> processWaitingShardsAndRemovedNodes(SnapshotsInProgress.Entry snapshotEntry, RoutingTable routingTable, DiscoveryNodes nodes, Predicate<String> nodeIdRemovalPredicate, Map<RepositoryShardId, SnapshotsInProgress.ShardSnapshotStatus> knownFailures) Walks through the snapshot entries' shard snapshots and creates applies updates from looking at removed nodes or indexes and known failed shard snapshots on the same shard IDs.- Parameters:
nodeIdRemovalPredicate- identify any nodes that are marked for removal / in shutdown modeknownFailures- already known failed shard snapshots, but more may be found in this method- Returns:
- an updated map of shard statuses
-
waitingShardsStartedOrUnassigned
public static boolean waitingShardsStartedOrUnassigned(SnapshotsInProgress snapshotsInProgress, ClusterChangedEvent event) -
removedNodesCleanupNeeded
public static boolean removedNodesCleanupNeeded(SnapshotsInProgress snapshotsInProgress, List<DiscoveryNode> removedNodes) -
onlySuccessfulFeatureStates
public static List<SnapshotFeatureInfo> onlySuccessfulFeatureStates(SnapshotsInProgress.Entry entry, List<String> finalIndices) Removes all feature states which have missing or failed shards, as they are no longer safely restorable.- Parameters:
entry- The "in progress" entry with a list of feature states and one or more failed shards.finalIndices- The final list of indices in the snapshot, after any indices that were concurrently deleted are removed.- Returns:
- The list of feature states which were completed successfully in the given entry.
-
readyDeletions
public static Tuple<ClusterState,List<SnapshotDeletionsInProgress.Entry>> readyDeletions(ClusterState currentState, @Nullable ProjectId projectId) Finds snapshot delete operations that are ready to execute in the givenClusterStateand computes a new cluster state that has all executable deletes marked as executing. Returns aTupleof the updated cluster state and all executable deletes. This can either beSnapshotDeletionsInProgress.Entrythat were already in stateSnapshotDeletionsInProgress.State.STARTEDor waiting entries in stateSnapshotDeletionsInProgress.State.WAITINGthat were moved toSnapshotDeletionsInProgress.State.STARTEDin the returned updated cluster state.- Parameters:
currentState- current cluster stateprojectId- the project for repositories where deletions should be prepared.nullmeans all projects- Returns:
- tuple of an updated cluster state and currently executable snapshot delete operations
-
stateWithoutSnapshot
public static ClusterState stateWithoutSnapshot(ClusterState state, Snapshot snapshot, FinalizeSnapshotContext.UpdatedShardGenerations updatedShardGenerations) Computes the cluster state resulting from removing a given snapshot create operation from the given state. This method will update the shard generations of snapshots that the given snapshot depended on so that finalizing them will not cause rolling back to an outdated shard generation.For example, shard snapshot X can be taken, but not finalized yet. Shard snapshot Y can then depend upon shard snapshot X. Then shard snapshot Y may finalize before shard snapshot X, but including X. However, X does not include Y. Therefore we update X to use Y's shard generation file (list of snapshots and dependencies) to avoid overwriting with X's file that is missing Y.
- Parameters:
state- current cluster statesnapshot- snapshot for which to remove the snapshot operation- Returns:
- updated cluster state
-
addSnapshotEntry
public static void addSnapshotEntry(List<SnapshotsInProgress.Entry> entries, SnapshotsInProgress.Entry entryToUpdate, @Nullable ImmutableOpenMap.Builder<ShardId, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments) -
addCloneEntry
public static void addCloneEntry(List<SnapshotsInProgress.Entry> entries, SnapshotsInProgress.Entry entryToUpdate, @Nullable ImmutableOpenMap.Builder<RepositoryShardId, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments) -
maybeAddUpdatedAssignment
@Nullable public static <T> ImmutableOpenMap.Builder<T,SnapshotsInProgress.ShardSnapshotStatus> maybeAddUpdatedAssignment(@Nullable ImmutableOpenMap.Builder<T, SnapshotsInProgress.ShardSnapshotStatus> updatedShardAssignments, SnapshotsInProgress.ShardSnapshotStatus finishedShardState, T shardId, Map<T, SnapshotsInProgress.ShardSnapshotStatus> statesToUpdate) -
deletionsWithoutSnapshots
@Nullable public static SnapshotDeletionsInProgress deletionsWithoutSnapshots(SnapshotDeletionsInProgress deletions, Collection<SnapshotId> snapshotIds, ProjectId projectId, String repository) Remove the givenSnapshotIds for the givenrepositoryfrom an instance ofSnapshotDeletionsInProgress. If no deletion contained any of the snapshot ids to remove then returnnull.- Parameters:
deletions- snapshot deletions to updatesnapshotIds- snapshot ids to removeprojectId- project for the repositoryrepository- repository that the snapshot ids belong to- Returns:
- updated
SnapshotDeletionsInProgressornullif unchanged
-
minCompatibleVersion
public static IndexVersion minCompatibleVersion(IndexVersion minNodeVersion, RepositoryData repositoryData, @Nullable Collection<SnapshotId> excluded) Determines the minimumIndexVersionthat the snapshot repository must be compatible with from the current nodes in the cluster and the contents of the repository. The minimum version is determined as the lowest version found across all snapshots in the repository and all nodes in the cluster.- Parameters:
minNodeVersion- minimum node version in the clusterrepositoryData- currentRepositoryDataof that repositoryexcluded- snapshot id to ignore when computing the minimum version (used to use newer metadata version after a snapshot delete)- Returns:
- minimum node version that must still be able to read the repository metadata
-
updateWithSnapshots
public static ClusterState updateWithSnapshots(ClusterState state, @Nullable SnapshotsInProgress snapshotsInProgress, @Nullable SnapshotDeletionsInProgress snapshotDeletionsInProgress) Shortcut to build newClusterStatefrom the current state and updated values ofSnapshotsInProgressandSnapshotDeletionsInProgress.- Parameters:
state- current cluster statesnapshotsInProgress- new value forSnapshotsInProgressornullif it's unchangedsnapshotDeletionsInProgress- new value forSnapshotDeletionsInProgressornullif it's unchanged- Returns:
- updated cluster state
-
failListenersIgnoringException
public static <T> void failListenersIgnoringException(@Nullable List<ActionListener<T>> listeners, Exception failure) -
completeListenersIgnoringException
public static <T> void completeListenersIgnoringException(@Nullable List<ActionListener<T>> listeners, T result) -
shards
public static ImmutableOpenMap<ShardId,SnapshotsInProgress.ShardSnapshotStatus> shards(SnapshotsInProgress snapshotsInProgress, SnapshotDeletionsInProgress deletionsInProgress, ProjectState currentState, Collection<IndexId> indices, boolean useShardGenerations, RepositoryData repositoryData, String repoName) Calculates the assignment of shards to data nodes for a new snapshot based on the given cluster state and the indices that should be included in the snapshot.- Parameters:
indices- Indices to snapshotuseShardGenerations- whether to writeShardGenerationsduring the snapshot- Returns:
- map of shard-id to snapshot-status of all shards included into current snapshot
-
initShardSnapshotStatus
public static SnapshotsInProgress.ShardSnapshotStatus initShardSnapshotStatus(ShardGeneration shardRepoGeneration, ShardRouting primary, Predicate<String> nodeIdRemovalPredicate) Compute the snapshot status for a given shard based on the current primary routing entry for the shard.- Parameters:
shardRepoGeneration- repository generation of the shard in the repositoryprimary- primary routing entry for the shardnodeIdRemovalPredicate- tests whether a node ID is currently marked for removal from the cluster- Returns:
- shard snapshot status
-
snapshottingDataStreams
public static Set<String> snapshottingDataStreams(ProjectState projectState, Set<String> dataStreamsToCheck) Returns the data streams that are currently being snapshotted (with partial == false) and that are contained in the indices-to-check set. -
snapshottingIndices
Returns the indices that are currently being snapshotted (with partial == false) and that are contained in the indices-to-check set. -
filterDataStreamAliases
public static Map<String,DataStreamAlias> filterDataStreamAliases(Map<String, DataStream> dataStreams, Map<String, DataStreamAlias> dataStreamAliases) Filters out the aliases that refer to data streams to do not exist in the provided data streams. Also rewrites the list of data streams an alias point to to only contain data streams that exist in the provided data streams.The purpose of this method is to capture the relevant data stream aliases based on the data streams that will be included in a snapshot.
- Parameters:
dataStreams- The provided data streams, which will be included in a snapshot.dataStreamAliases- The data streams aliases that may contain aliases that refer to data streams that don't exist in the provided data streams.- Returns:
- The filtered data streams aliases only referring to data streams in the provided data streams.
-
logSnapshotFailure
-
snapshotFailureLogLevel
-