- All Implemented Interfaces:
Closeable,AutoCloseable
- Direct Known Subclasses:
InternalEngine,ReadOnlyEngine
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic classstatic classstatic interfacestatic final recordstatic classstatic classstatic classstatic interfacestatic classstatic classprotected static final classA throttling class that can be activated, causing theacquireThrottlemethod to block on a lock when throttling is enabledstatic classprotected static final classA Lock implementation that always allows the lock to be acquiredstatic classstatic classstatic final recordCaptures the result of a refresh operation on the index shard.static classBase class for index and delete operation results Holds result meta data (e.g.static final classstatic enumstatic classstatic interfacestatic interfaceCalled for each new opened engine reader to warm new segments -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final Stringprotected static final Stringprotected final booleanprotected final EngineConfigstatic final Stringprotected final Engine.EventListenerprotected final org.apache.lucene.util.SetOnce<Exception> protected final ReentrantLockstatic final Stringstatic final Stringprotected final AtomicBooleanprotected longprotected final org.apache.logging.log4j.Loggerstatic final Stringstatic final Stringstatic final Stringstatic final Stringprotected final ShardIdprotected final Storestatic final Stringstatic final long -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionprotected final Releasableabstract CloseableAcquires a lock on the translog files and Lucene soft-deleted documents to prevent them from being trimmedfinal Engine.IndexCommitRefAcquires the index commit that should be included in a snapshot.abstract Engine.IndexCommitRefacquireLastIndexCommit(boolean flushFirst) Snapshots the most recent index and returns a handle to it.abstract Engine.IndexCommitRefSnapshots the most recent safe index commit from the engine.final Engine.SearcheracquireSearcher(String source) acquireSearcher(String source, Engine.SearcherScope scope) acquireSearcher(String source, Engine.SearcherScope scope, Function<Engine.Searcher, Engine.Searcher> wrapper) final Engine.SearcherSupplierAcquires a point-in-time reader that can be used to createEngine.Searchers on demand.acquireSearcherSupplier(Function<Engine.Searcher, Engine.Searcher> wrapper, Engine.SearcherScope scope) Acquires a point-in-time reader that can be used to createEngine.Searchers on demand.abstract voidRequest that this engine throttle incoming indexing requests to one thread.voidaddFlushListener(Translog.Location location, ActionListener<Long> listener) voidaddPrimaryTermAndGenerationListener(long minPrimaryTerm, long minGeneration, ActionListener<Long> listener) Allows registering a listener for when the index shard is on a primary term >= minPrimaryTerm and a segment generation >= minGeneration.voidaddSegmentGenerationListener(long minGeneration, ActionListener<Long> listener) Deprecated.abstract voidadvanceMaxSeqNoOfUpdatesOrDeletes(long maxSeqNoOfUpdatesOnPrimary) A replica shard receives a new max_seq_no_of_updates from its primary shard, then calls this method to advance this marker to at least the given sequence number.booleanWhether search idleness may be allowed to be considered for skipping a scheduled refresh.abstract voidasyncEnsureGlobalCheckpointSynced(long globalCheckpoint, Consumer<Exception> listener) Ensures that the global checkpoint has been persisted to the underlying storage.abstract voidasyncEnsureTranslogSynced(Translog.Location location, Consumer<Exception> listener) Ensures that the location has been written to the underlying storage.voidclose()protected abstract voidcloseNoLock(String reason, CountDownLatch closedLatch) Closes the engine without acquiring any refs or locks.final CommitStatsget commits stats for the last commitabstract CompletionStatscompletionStats(String... fieldNamePatterns) Returns theCompletionStatsfor this enginefinal EngineConfigconfig()abstract intcountChanges(String source, long fromSeqNo, long toSeqNo) Counts the number of operations in the range of the given sequence numbers.abstract voidReverses a previousactivateThrottling()call.abstract Engine.DeleteResultdelete(Engine.Delete delete) Perform document delete operation on the engineprotected final DenseVectorStatsdenseVectorStats(org.apache.lucene.index.IndexReader indexReader, List<String> fields) denseVectorStats(MappingLookup mappingLookup) Returns theDenseVectorStatsfor this engineprotected final DocsStatsdocsStats(org.apache.lucene.index.IndexReader indexReader) docStats()Returns theDocsStatsfor this engineprotected final voidprotected final voidensureOpen(Exception suppressed) voidexternalRefresh(String source, ActionListener<Engine.RefreshResult> listener) An async variant ofrefresh(String)that may apply some rate-limiting.voidfailEngine(String reason, Exception failure) fail engine due to some error.protected voidfillSegmentStats(org.apache.lucene.index.SegmentReader segmentReader, boolean includeSegmentFileSizes, SegmentsStats stats) abstract intfillSeqNoGaps(long primaryTerm) Fills up the local checkpoints history with no-ops until the local checkpoint and the max seen sequence ID are identical.final voidflush()Flushes the state of the engine including the transaction log, clearing memory and persisting documents in the lucene index to disk including a potentially heavy and durable fsync operation.voidflush(boolean force, boolean waitIfOngoing) A Blocking helper method for calling the async flush method.final voidflush(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) Flushes the state of the engine including the transaction log, clearing memory, and writing the documents in the Lucene index to disk.voidFlush the engine (committing segments to disk and truncating the translog) and close it.protected abstract voidflushHoldingLock(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) The actual implementation offlush(boolean, boolean, ActionListener), to be called either when holding a ref that ensures the engine remains open, or holdingIndexShard#engineMutexwhile closing the engine.abstract voidforceMerge(boolean flush, int maxNumSegments, boolean onlyExpungeDeletes, String forceMergeUUID) Triggers a forced merge on this engineabstract Engine.GetResultget(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) final EngineConfigprotected final Engine.GetResultgetFromSearcher(Engine.Get get, Engine.Searcher searcher, boolean uncachedLookup) getFromTranslog(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) Similar toget(org.elasticsearch.index.engine.Engine.Get, org.elasticsearch.index.mapper.MappingLookup, org.elasticsearch.index.mapper.DocumentParser, java.util.function.Function<org.elasticsearch.index.engine.Engine.Searcher, org.elasticsearch.index.engine.Engine.Searcher>), but it only attempts to serve the get from the translog.abstract Stringreturns the history uuid for the engineabstract longHow much heap is used that would be freed by a refresh.abstract longReturns the number of milliseconds this engine was under index throttling.abstract org.apache.lucene.index.SegmentInfosabstract longReturns the latest global checkpoint value that has been persisted in the underlying storage (i.e.longReturns the timestamp of the last write in nanoseconds.longReturns the maximum auto_id_timestamp of all append-only index requests have been processed by this engine or the auto_id_timestamp received from its primary shard viaupdateMaxUnsafeAutoIdTimestamp(long).abstract longabstract longReturns the maximum sequence number of either update or delete operations have been processed in this engine or the sequence number fromadvanceMaxSeqNoOfUpdatesOrDeletes(long).abstract longGets the minimum retained sequence number for this engine.abstract longabstract longabstract ShardLongFieldRangegetRawFieldRange(String field) protected abstract org.apache.lucene.search.ReferenceManager<ElasticsearchDirectoryReader> abstract SafeCommitInfoabstract SeqNoStatsgetSeqNoStats(long globalCheckpoint) longReturns the total time flushes have been executed excluding waiting on locks.abstract Translog.LocationReturns the last location that the translog of this engine has written into.abstract TranslogStatsabstract longReturns how many bytes we are currently moving from heap to diskabstract booleanhasCompleteOperationHistory(String reason, long startingSeqNo) Checks if this engine has every operations sincestartingSeqNo(inclusive) in its history (either Lucene or translog)abstract Engine.IndexResultindex(Engine.Index index) Perform document index operation on the engineprotected final booleanprotected final booleanabstract booleanReturns thetrueiff this engine is currently under index throttling.abstract booleanChecks if the underlying storage sync is required.protected booleanmaybeFailEngine(String source, Exception e) Check whether the engine should be failedabstract voidTries to prune buffered deletes from the version map.abstract voidmaybeRefresh(String source, ActionListener<Engine.RefreshResult> listener) Asynchronously refreshes the engine for new search operations to reflect the latest changes unless another thread is already refreshing the engine concurrently.abstract Translog.SnapshotnewChangesSnapshot(String source, long fromSeqNo, long toSeqNo, boolean requiredFullRange, boolean singleConsumer, boolean accessStats, long maxChunkSize) Creates a new history snapshot from Lucene for reading operations whose seqno in the requesting seqno range (both inclusive).abstract Engine.NoOpResultnoOp(Engine.NoOp noOp) voidstatic IndexVersionreadIndexVersion(String esVersion) Reads anIndexVersionfrom anes_versionmetadata stringfinal voidrecoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo) Performs recovery from the transaction log up torecoverUpToSeqNo(inclusive).abstract voidrecoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo, ActionListener<Void> listener) Performs recovery from the transaction log up torecoverUpToSeqNo(inclusive).abstract Engine.RefreshResultSynchronously refreshes the engine for new search operations to reflect the latest changes.booleanabstract intrestoreLocalHistoryFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner) This method replays translog to restore the Lucene index which might be reverted previously.abstract voidRolls the translog generation and cleans unneeded.segments()The list of segments in the engine.segments(boolean includeVectorFormatsInfo) segmentsStats(boolean includeSegmentFileSizes, boolean includeUnloadedSegments) Global stats on segments.abstract booleanChecks if this engine should be flushed periodically.abstract booleanTests whether or not the translog generation should be rolled to a new generation.abstract voidDo not replay translog operations, but make the engine be ready.protected final SparseVectorStatssparseVectorStats(org.apache.lucene.index.IndexReader indexReader, List<org.apache.lucene.util.BytesRef> fields) sparseVectorStats(MappingLookup mappingLookup) Returns theSparseVectorStatsfor this engineabstract voidabstract voidtrimOperationsFromTranslog(long belowTerm, long aboveSeqNo) Trims translog for terms belowbelowTermand seq# aboveaboveSeqNoabstract voidchecks and removes translog files that no longer need to be retained.abstract voidupdateMaxUnsafeAutoIdTimestamp(long newTimestamp) Forces this engine to advance its max_unsafe_auto_id_timestamp marker to at least the given timestamp.voidPerforms the pre-closing checks on theEngine.abstract voidCalled when our engine is using too much heap and should move buffered indexed/deleted documents to disk.protected voidwriterSegmentStats(SegmentsStats stats)
-
Field Details
-
SYNC_COMMIT_ID
- See Also:
-
HISTORY_UUID_KEY
- See Also:
-
FORCE_MERGE_UUID_KEY
- See Also:
-
MIN_RETAINED_SEQNO
- See Also:
-
MAX_UNSAFE_AUTO_ID_TIMESTAMP_COMMIT_ID
- See Also:
-
ES_VERSION
- See Also:
-
SEARCH_SOURCE
- See Also:
-
CAN_MATCH_SEARCH_SOURCE
- See Also:
-
DOC_STATS_SOURCE
- See Also:
-
UNKNOWN_PRIMARY_TERM
public static final long UNKNOWN_PRIMARY_TERM- See Also:
-
ROOT_DOC_FIELD_NAME
- See Also:
-
shardId
-
logger
protected final org.apache.logging.log4j.Logger logger -
engineConfig
-
store
-
isClosed
-
eventListener
-
failEngineLock
-
failedEngine
-
enableRecoverySource
protected final boolean enableRecoverySource -
lastWriteNanos
protected volatile long lastWriteNanos
-
-
Constructor Details
-
Engine
-
-
Method Details
-
readIndexVersion
Reads anIndexVersionfrom anes_versionmetadata string -
config
-
getLastCommittedSegmentInfos
public abstract org.apache.lucene.index.SegmentInfos getLastCommittedSegmentInfos() -
getMergeStats
-
getHistoryUUID
returns the history uuid for the engine -
getWritingBytes
public abstract long getWritingBytes()Returns how many bytes we are currently moving from heap to disk -
completionStats
Returns theCompletionStatsfor this engine -
docStats
Returns theDocsStatsfor this engine -
docsStats
-
denseVectorStats
Returns theDenseVectorStatsfor this engine -
denseVectorStats
protected final DenseVectorStats denseVectorStats(org.apache.lucene.index.IndexReader indexReader, List<String> fields) -
sparseVectorStats
Returns theSparseVectorStatsfor this engine -
sparseVectorStats
protected final SparseVectorStats sparseVectorStats(org.apache.lucene.index.IndexReader indexReader, List<org.apache.lucene.util.BytesRef> fields) -
verifyEngineBeforeIndexClosing
Performs the pre-closing checks on theEngine.- Throws:
IllegalStateException- if the sanity checks failed
-
getIndexThrottleTimeInMillis
public abstract long getIndexThrottleTimeInMillis()Returns the number of milliseconds this engine was under index throttling. -
isThrottled
public abstract boolean isThrottled()Returns thetrueiff this engine is currently under index throttling.- See Also:
-
trimOperationsFromTranslog
public abstract void trimOperationsFromTranslog(long belowTerm, long aboveSeqNo) throws EngineException Trims translog for terms belowbelowTermand seq# aboveaboveSeqNo- Throws:
EngineException- See Also:
-
getTotalFlushTimeExcludingWaitingOnLockInMillis
public long getTotalFlushTimeExcludingWaitingOnLockInMillis()Returns the total time flushes have been executed excluding waiting on locks. -
index
Perform document index operation on the engine- Parameters:
index- operation to perform- Returns:
Engine.IndexResultcontaining updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown- Throws:
IOException
-
delete
Perform document delete operation on the engine- Parameters:
delete- operation to perform- Returns:
Engine.DeleteResultcontaining updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown- Throws:
IOException
-
noOp
- Throws:
IOException
-
getFromSearcher
protected final Engine.GetResult getFromSearcher(Engine.Get get, Engine.Searcher searcher, boolean uncachedLookup) throws EngineException - Throws:
EngineException
-
get
public abstract Engine.GetResult get(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) -
getFromTranslog
public Engine.GetResult getFromTranslog(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) Similar toget(org.elasticsearch.index.engine.Engine.Get, org.elasticsearch.index.mapper.MappingLookup, org.elasticsearch.index.mapper.DocumentParser, java.util.function.Function<org.elasticsearch.index.engine.Engine.Searcher, org.elasticsearch.index.engine.Engine.Searcher>), but it only attempts to serve the get from the translog. If not found in translog, it returns null, asEngine.GetResult.NOT_EXISTScould mean deletion. -
acquireSearcherSupplier
public final Engine.SearcherSupplier acquireSearcherSupplier(Function<Engine.Searcher, Engine.Searcher> wrapper) throws EngineExceptionAcquires a point-in-time reader that can be used to createEngine.Searchers on demand.- Throws:
EngineException
-
acquireSearcherSupplier
public Engine.SearcherSupplier acquireSearcherSupplier(Function<Engine.Searcher, Engine.Searcher> wrapper, Engine.SearcherScope scope) throws EngineExceptionAcquires a point-in-time reader that can be used to createEngine.Searchers on demand.- Throws:
EngineException
-
acquireSearcher
- Throws:
EngineException
-
acquireSearcher
public Engine.Searcher acquireSearcher(String source, Engine.SearcherScope scope) throws EngineException - Throws:
EngineException
-
acquireSearcher
public Engine.Searcher acquireSearcher(String source, Engine.SearcherScope scope, Function<Engine.Searcher, Engine.Searcher> wrapper) throws EngineException- Throws:
EngineException
-
getReferenceManager
protected abstract org.apache.lucene.search.ReferenceManager<ElasticsearchDirectoryReader> getReferenceManager(Engine.SearcherScope scope) -
isTranslogSyncNeeded
public abstract boolean isTranslogSyncNeeded()Checks if the underlying storage sync is required. -
allowSearchIdleOptimization
public boolean allowSearchIdleOptimization()Whether search idleness may be allowed to be considered for skipping a scheduled refresh. -
asyncEnsureTranslogSynced
public abstract void asyncEnsureTranslogSynced(Translog.Location location, Consumer<Exception> listener) Ensures that the location has been written to the underlying storage. -
asyncEnsureGlobalCheckpointSynced
public abstract void asyncEnsureGlobalCheckpointSynced(long globalCheckpoint, Consumer<Exception> listener) Ensures that the global checkpoint has been persisted to the underlying storage. -
syncTranslog
- Throws:
IOException
-
acquireHistoryRetentionLock
Acquires a lock on the translog files and Lucene soft-deleted documents to prevent them from being trimmed -
countChanges
Counts the number of operations in the range of the given sequence numbers.- Parameters:
source- the source of the requestfromSeqNo- the start sequence number (inclusive)toSeqNo- the end sequence number (inclusive)- Throws:
IOException- See Also:
-
newChangesSnapshot
public abstract Translog.Snapshot newChangesSnapshot(String source, long fromSeqNo, long toSeqNo, boolean requiredFullRange, boolean singleConsumer, boolean accessStats, long maxChunkSize) throws IOException Creates a new history snapshot from Lucene for reading operations whose seqno in the requesting seqno range (both inclusive). This feature requires soft-deletes enabled. If soft-deletes are disabled, this method will throw anIllegalStateException.- Throws:
IOException
-
hasCompleteOperationHistory
Checks if this engine has every operations sincestartingSeqNo(inclusive) in its history (either Lucene or translog) -
getMinRetainedSeqNo
public abstract long getMinRetainedSeqNo()Gets the minimum retained sequence number for this engine.- Returns:
- the minimum retained sequence number
-
getTranslogStats
-
getTranslogLastWriteLocation
Returns the last location that the translog of this engine has written into. -
ensureOpen
-
ensureOpen
protected final void ensureOpen() -
commitStats
get commits stats for the last commit -
getMaxSeqNo
public abstract long getMaxSeqNo()- Returns:
- the max issued or seen seqNo for this Engine
-
getProcessedLocalCheckpoint
public abstract long getProcessedLocalCheckpoint()- Returns:
- the processed local checkpoint for this Engine
-
getPersistedLocalCheckpoint
public abstract long getPersistedLocalCheckpoint()- Returns:
- the persisted local checkpoint for this Engine
-
getSeqNoStats
- Returns:
- a
SeqNoStatsobject, using local state and the supplied global checkpoint
-
getLastSyncedGlobalCheckpoint
public abstract long getLastSyncedGlobalCheckpoint()Returns the latest global checkpoint value that has been persisted in the underlying storage (i.e. translog's checkpoint) -
segmentsStats
public SegmentsStats segmentsStats(boolean includeSegmentFileSizes, boolean includeUnloadedSegments) Global stats on segments. -
fillSegmentStats
protected void fillSegmentStats(org.apache.lucene.index.SegmentReader segmentReader, boolean includeSegmentFileSizes, SegmentsStats stats) -
writerSegmentStats
-
getIndexBufferRAMBytesUsed
public abstract long getIndexBufferRAMBytesUsed()How much heap is used that would be freed by a refresh. This includes both the current memory being freed and any remaining memory usage that could be freed, e.g., by refreshing. Note that this may throwAlreadyClosedException. -
segments
The list of segments in the engine. -
segments
-
refreshNeeded
public boolean refreshNeeded() -
refresh
Synchronously refreshes the engine for new search operations to reflect the latest changes.- Throws:
EngineException
-
externalRefresh
An async variant ofrefresh(String)that may apply some rate-limiting. -
maybeRefresh
@Nullable public abstract void maybeRefresh(String source, ActionListener<Engine.RefreshResult> listener) throws EngineException Asynchronously refreshes the engine for new search operations to reflect the latest changes unless another thread is already refreshing the engine concurrently.- Throws:
EngineException
-
writeIndexingBuffer
Called when our engine is using too much heap and should move buffered indexed/deleted documents to disk.- Throws:
IOException
-
shouldPeriodicallyFlush
public abstract boolean shouldPeriodicallyFlush()Checks if this engine should be flushed periodically. This check is mainly based on the uncommitted translog size and the translog flush threshold setting. -
flush
A Blocking helper method for calling the async flush method.- Throws:
EngineException
-
flush
public final void flush(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) throws EngineException Flushes the state of the engine including the transaction log, clearing memory, and writing the documents in the Lucene index to disk. This method will synchronously flush on the calling thread. However, depending on engine implementation, full durability will not be guaranteed until the listener is triggered.- Parameters:
force- iftruea lucene commit is executed even if no changes need to be committed.waitIfOngoing- iftruethis call will block until all currently running flushes have finished. Otherwise this call will return without blocking.listener- to notify after fully durability has been achieved. ifwaitIfOngoing==falseand ongoing request is detected, no flush will have occurred and the listener will be completed with a marker indicating no flush and unknown generation.- Throws:
EngineException
-
flushHoldingLock
protected abstract void flushHoldingLock(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) throws EngineException The actual implementation offlush(boolean, boolean, ActionListener), to be called either when holding a ref that ensures the engine remains open, or holdingIndexShard#engineMutexwhile closing the engine.- Throws:
EngineException
-
flush
Flushes the state of the engine including the transaction log, clearing memory and persisting documents in the lucene index to disk including a potentially heavy and durable fsync operation. This operation is not going to block if another flush operation is currently running and won't write a lucene commit if nothing needs to be committed.- Throws:
EngineException
-
trimUnreferencedTranslogFiles
checks and removes translog files that no longer need to be retained. SeeTranslogDeletionPolicyfor details- Throws:
EngineException
-
shouldRollTranslogGeneration
public abstract boolean shouldRollTranslogGeneration()Tests whether or not the translog generation should be rolled to a new generation. This test is based on the size of the current generation compared to the configured generation threshold size.- Returns:
trueif the current generation should be rolled to a new generation
-
rollTranslogGeneration
Rolls the translog generation and cleans unneeded.- Throws:
EngineException
-
forceMerge
public abstract void forceMerge(boolean flush, int maxNumSegments, boolean onlyExpungeDeletes, String forceMergeUUID) throws EngineException, IOException Triggers a forced merge on this engine- Throws:
EngineExceptionIOException
-
acquireLastIndexCommit
public abstract Engine.IndexCommitRef acquireLastIndexCommit(boolean flushFirst) throws EngineException Snapshots the most recent index and returns a handle to it. If needed will try and "commit" the lucene index to make sure we have a "fresh" copy of the files to snapshot.- Parameters:
flushFirst- indicates whether the engine should flush before returning the snapshot- Throws:
EngineException
-
acquireSafeIndexCommit
Snapshots the most recent safe index commit from the engine.- Throws:
EngineException
-
acquireIndexCommitForSnapshot
Acquires the index commit that should be included in a snapshot.- Throws:
EngineException
-
getSafeCommitInfo
- Returns:
- a summary of the contents of the current safe commit
-
failEngine
fail engine due to some error. the engine will also be closed. The underlying store is marked corrupted iff failure is caused by index corruption -
maybeFailEngine
Check whether the engine should be failed -
closeNoLock
Closes the engine without acquiring any refs or locks. The caller should either have changedisClosingfromfalsetotrueor else must hold thefailEngineLock. The implementation must decrement the supplied latch when done. -
isDrainedForClose
protected final boolean isDrainedForClose() -
isClosing
protected final boolean isClosing() -
acquireEnsureOpenRef
-
flushAndClose
Flush the engine (committing segments to disk and truncating the translog) and close it.- Throws:
IOException
-
close
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Throws:
IOException
-
onSettingsChanged
public void onSettingsChanged() -
getLastWriteNanos
public long getLastWriteNanos()Returns the timestamp of the last write in nanoseconds. Note: this time might not be absolutely accurate since theEngine.Operation.startTime()is used which might be slightly inaccurate.- See Also:
-
activateThrottling
public abstract void activateThrottling()Request that this engine throttle incoming indexing requests to one thread. Must be matched by a later call todeactivateThrottling(). -
deactivateThrottling
public abstract void deactivateThrottling()Reverses a previousactivateThrottling()call. -
restoreLocalHistoryFromTranslog
public abstract int restoreLocalHistoryFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner) throws IOException This method replays translog to restore the Lucene index which might be reverted previously. This ensures that all acknowledged writes are restored correctly when this engine is promoted.- Returns:
- the number of translog operations have been recovered
- Throws:
IOException
-
fillSeqNoGaps
Fills up the local checkpoints history with no-ops until the local checkpoint and the max seen sequence ID are identical.- Parameters:
primaryTerm- the shards primary term this engine was created for- Returns:
- the number of no-ops added
- Throws:
IOException
-
recoverFromTranslog
public final void recoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo) throws IOException Performs recovery from the transaction log up torecoverUpToSeqNo(inclusive). This operation will close the engine if the recovery fails. Use EngineTestCase#recoverFromTranslog for test usages- Parameters:
translogRecoveryRunner- the translog recovery runnerrecoverUpToSeqNo- the upper bound, inclusive, of sequence number to be recovered- Throws:
IOException
-
recoverFromTranslog
public abstract void recoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo, ActionListener<Void> listener) Performs recovery from the transaction log up torecoverUpToSeqNo(inclusive). This operation will close the engine if the recovery fails.- Parameters:
translogRecoveryRunner- the translog recovery runnerrecoverUpToSeqNo- the upper bound, inclusive, of sequence number to be recoveredlistener- listener notified on completion of the recovery, whether successful or otherwise
-
skipTranslogRecovery
public abstract void skipTranslogRecovery()Do not replay translog operations, but make the engine be ready. -
maybePruneDeletes
public abstract void maybePruneDeletes()Tries to prune buffered deletes from the version map. -
getMaxSeenAutoIdTimestamp
public long getMaxSeenAutoIdTimestamp()Returns the maximum auto_id_timestamp of all append-only index requests have been processed by this engine or the auto_id_timestamp received from its primary shard viaupdateMaxUnsafeAutoIdTimestamp(long). Notes this method returns the auto_id_timestamp of all append-only requests, not max_unsafe_auto_id_timestamp. -
updateMaxUnsafeAutoIdTimestamp
public abstract void updateMaxUnsafeAutoIdTimestamp(long newTimestamp) Forces this engine to advance its max_unsafe_auto_id_timestamp marker to at least the given timestamp. The engine will disable optimization for all append-only whose timestamp at mostnewTimestamp. -
getMaxSeqNoOfUpdatesOrDeletes
public abstract long getMaxSeqNoOfUpdatesOrDeletes()Returns the maximum sequence number of either update or delete operations have been processed in this engine or the sequence number fromadvanceMaxSeqNoOfUpdatesOrDeletes(long). An index request is considered as an update operation if it overwrites the existing documents in Lucene index with the same document id.A note on the optimization using max_seq_no_of_updates_or_deletes: For each operation O, the key invariants are:
- I1: There is no operation on docID(O) with seqno that is > MSU(O) and < seqno(O)
- I2: If MSU(O) < seqno(O) then docID(O) did not exist when O was applied; more precisely, if there is any O' with seqno(O') < seqno(O) and docID(O') = docID(O) then the one with the greatest seqno is a delete.
When a receiving shard (either a replica or a follower) receives an operation O, it must first ensure its own MSU at least MSU(O), and then compares its MSU to its local checkpoint (LCP). If LCP < MSU then there's a gap: there may be some operations that act on docID(O) about which we do not yet know, so we cannot perform an add. Note this also covers the case where a future operation O' with seqNo(O') > seqNo(O) and docId(O') = docID(O) is processed before O. In that case MSU(O') is at least seqno(O') and this means MSU >= seqNo(O') > seqNo(O) > LCP (because O wasn't processed yet).
However, if MSU <= LCP then there is no gap: we have processed every operation <= LCP, and no operation O' with seqno(O') > LCP and seqno(O') < seqno(O) also has docID(O') = docID(O), because such an operation would have seqno(O') > LCP >= MSU >= MSU(O) which contradicts the first invariant. Furthermore in this case we immediately know that docID(O) has been deleted (or never existed) without needing to check Lucene for the following reason. If there's no earlier operation on docID(O) then this is clear, so suppose instead that the preceding operation on docID(O) is O': 1. The first invariant above tells us that seqno(O') <= MSU(O) <= LCP so we have already applied O' to Lucene. 2. Also MSU(O) <= MSU <= LCP < seqno(O) (we discard O if seqno(O) <= LCP) so the second invariant applies, meaning that the O' was a delete.
Therefore, if MSU <= LCP < seqno(O) we know that O can safely be optimized with and added to lucene with addDocument. Moreover, operations that are optimized using the MSU optimization must not be processed twice as this will create duplicates in Lucene. To avoid this we check the local checkpoint tracker to see if an operation was already processed.
- See Also:
-
advanceMaxSeqNoOfUpdatesOrDeletes
public abstract void advanceMaxSeqNoOfUpdatesOrDeletes(long maxSeqNoOfUpdatesOnPrimary) A replica shard receives a new max_seq_no_of_updates from its primary shard, then calls this method to advance this marker to at least the given sequence number. -
getRawFieldRange
- Returns:
- a
ShardLongFieldRangecontaining the min and max raw values of the given field for this shard if the engine guarantees these values never to change, orShardLongFieldRange.EMPTYif this field is empty, orShardLongFieldRange.UNKNOWNif this field's value range may change in future. - Throws:
IOException
-
getEngineConfig
-
addSegmentGenerationListener
@Deprecated public void addSegmentGenerationListener(long minGeneration, ActionListener<Long> listener) Deprecated.Allows registering a listener for when the index shard is on a segment generation >= minGeneration. -
addPrimaryTermAndGenerationListener
public void addPrimaryTermAndGenerationListener(long minPrimaryTerm, long minGeneration, ActionListener<Long> listener) Allows registering a listener for when the index shard is on a primary term >= minPrimaryTerm and a segment generation >= minGeneration. -
addFlushListener
-
addPrimaryTermAndGenerationListener(long, long, ActionListener)instead.