- All Implemented Interfaces:
Closeable,AutoCloseable
- Direct Known Subclasses:
NoOpEngine
- See Also:
-
Nested Class Summary
Nested classes/interfaces inherited from class org.elasticsearch.index.engine.Engine
Engine.Delete, Engine.DeleteResult, Engine.EventListener, Engine.FlushResult, Engine.Get, Engine.GetResult, Engine.Index, Engine.IndexCommitListener, Engine.IndexCommitRef, Engine.IndexResult, Engine.IndexThrottle, Engine.NoOp, Engine.NoOpLock, Engine.NoOpResult, Engine.Operation, Engine.RefreshResult, Engine.Result, Engine.Searcher, Engine.SearcherScope, Engine.SearcherSupplier, Engine.TranslogRecoveryRunner, Engine.Warmer -
Field Summary
FieldsFields inherited from class org.elasticsearch.index.engine.Engine
CAN_MATCH_SEARCH_SOURCE, DOC_STATS_SOURCE, enableRecoverySource, engineConfig, ES_VERSION, eventListener, failedEngine, failEngineLock, FORCE_MERGE_UUID_KEY, HISTORY_UUID_KEY, isClosed, lastWriteNanos, logger, MAX_UNSAFE_AUTO_ID_TIMESTAMP_COMMIT_ID, MIN_RETAINED_SEQNO, pauseIndexingOnThrottle, ROOT_DOC_FIELD_NAME, SEARCH_SOURCE, SEGMENTS_STATS_SOURCE, shardId, store, UNKNOWN_PRIMARY_TERM -
Constructor Summary
ConstructorsConstructorDescriptionReadOnlyEngine(EngineConfig config, SeqNoStats seqNoStats, TranslogStats translogStats, boolean obtainLock, Function<org.apache.lucene.index.DirectoryReader, org.apache.lucene.index.DirectoryReader> readerWrapperFunction, boolean requireCompleteHistory, boolean lazilyLoadSoftDeletes) Creates a new ReadOnlyEngine. -
Method Summary
Modifier and TypeMethodDescriptionAcquires a lock on Lucene soft-deleted documents to prevent them from being trimmedacquireLastIndexCommit(boolean flushFirst) Snapshots the most recent index and returns a handle to it.Snapshots the most recent safe index commit from the engine.acquireSearcherSupplier(Function<Engine.Searcher, Engine.Searcher> wrapper, Engine.SearcherScope scope) Acquires a point-in-time reader that can be used to createEngine.Searchers on demand.voidRequest that this engine throttle incoming indexing requests to one thread.voidadvanceMaxSeqNoOfUpdatesOrDeletes(long maxSeqNoOfUpdatesOnPrimary) A replica shard receives a new max_seq_no_of_updates from its primary shard, then calls this method to advance this marker to at least the given sequence number.protected booleanassertMaxSeqNoEqualsToGlobalCheckpoint(long maxSeqNo, long globalCheckpoint) voidasyncEnsureGlobalCheckpointSynced(long globalCheckpoint, Consumer<Exception> listener) Ensures that the global checkpoint has been persisted to the underlying storage.voidasyncEnsureTranslogSynced(Translog.Location location, Consumer<Exception> listener) Ensures that the location has been written to the underlying storage.protected voidcloseNoLock(String reason, CountDownLatch closedLatch) Closes the engine without acquiring any refs or locks.completionStats(String... fieldNamePatterns) Returns theCompletionStatsfor this engineintcountChanges(String source, long fromSeqNo, long toSeqNo) Counts the number of operations in the range of the given sequence numbers.voidReverses a previousEngine.activateThrottling()call.delete(Engine.Delete delete) Perform document delete operation on the engineprotected voidensureMaxSeqNoEqualsToGlobalCheckpoint(SeqNoStats seqNoStats) intfillSeqNoGaps(long primaryTerm) Fills up the local checkpoints history with no-ops until the local checkpoint and the max seen sequence ID are identical.protected voidflushHoldingLock(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) The actual implementation ofEngine.flush(boolean, boolean, ActionListener), to be called either when holding a ref that ensures the engine remains open, or holdingIndexShard#engineMutexwhile closing the engine.voidforceMerge(boolean flush, int maxNumSegments, boolean onlyExpungeDeletes, String forceMergeUUID) Triggers a forced merge on this engineget(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) final Stringreturns the history uuid for the enginelongHow much heap is used that would be freed by a refresh.longReturns the number of milliseconds this engine was under index throttling.org.apache.lucene.index.SegmentInfoslongReturns the latest global checkpoint value that has been persisted in the underlying storage (i.e.longlongReturns the maximum sequence number of either update or delete operations have been processed in this engine or the sequence number fromEngine.advanceMaxSeqNoOfUpdatesOrDeletes(long).longGets the minimum retained sequence number for this engine.longlonggetRawFieldRange(String field) protected org.apache.lucene.search.ReferenceManager<ElasticsearchDirectoryReader> getSeqNoStats(long globalCheckpoint) Returns the last location that the translog of this engine has written into.longReturns how many bytes we are currently moving from heap to diskbooleanhasCompleteOperationHistory(String reason, long startingSeqNo) Checks if this engine has every operations sincestartingSeqNo(inclusive) in its history (either Lucene or translog)index(Engine.Index index) Perform document index operation on the enginebooleanReturns thetrueiff this engine is currently under index throttling.booleanChecks if the underlying storage sync is required.voidTries to prune buffered deletes from the version map.voidmaybeRefresh(String source, ActionListener<Engine.RefreshResult> listener) Asynchronously refreshes the engine for new search operations to reflect the latest changes unless another thread is already refreshing or reseting the engine concurrently.newChangesSnapshot(String source, long fromSeqNo, long toSeqNo, boolean requiredFullRange, boolean singleConsumer, boolean accessStats, long maxChunkSize) Creates a new history snapshot from Lucene for reading operations whose seqno in the requesting seqno range (both inclusive).noOp(Engine.NoOp noOp) protected org.apache.lucene.index.DirectoryReaderopen(org.apache.lucene.index.IndexCommit commit) protected org.apache.lucene.index.DirectoryReaderopenDirectory(org.apache.lucene.store.Directory directory) voidrecoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo, ActionListener<Void> listener) Performs recovery from the transaction log up torecoverUpToSeqNo(inclusive).Synchronously refreshes the engine for new search operations to reflect the latest changes.booleanintrestoreLocalHistoryFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner) This method replays translog to restore the Lucene index which might be reverted previously.voidReverses a previousEngine.suspendThrottling()call.voidRolls the translog generation and cleans unneeded.segments()The list of segments in the engine.segments(boolean includeVectorFormatsInfo) booleanChecks if this engine should be flushed periodically.booleanTests whether or not the translog generation should be rolled to a new generation.voidDo not replay translog operations, but make the engine be ready.voidIf indexing is throttled to the point where it is paused completely, another task trying to get indexing permits might want to pause throttling by letting one thread pass at a time so that it does not get starved.voidvoidtrimOperationsFromTranslog(long belowTerm, long aboveSeqNo) Trims translog for terms belowbelowTermand seq# aboveaboveSeqNovoidchecks and removes translog files that no longer need to be retained.voidupdateMaxUnsafeAutoIdTimestamp(long newTimestamp) Forces this engine to advance its max_unsafe_auto_id_timestamp marker to at least the given timestamp.voidPerforms the pre-closing checks on theEngine.protected final ElasticsearchDirectoryReaderwrapReader(org.apache.lucene.index.DirectoryReader reader, Function<org.apache.lucene.index.DirectoryReader, org.apache.lucene.index.DirectoryReader> readerWrapperFunction, ESCacheHelper esCacheHelper) voidCalled when our engine is using too much heap and should move buffered indexed/deleted documents to disk.Methods inherited from class org.elasticsearch.index.engine.Engine
acquireEnsureOpenRef, acquireIndexCommitForSnapshot, acquireSearcher, acquireSearcher, acquireSearcher, acquireSearcherSupplier, addFlushListener, addPrimaryTermAndGenerationListener, addSegmentGenerationListener, allowSearchIdleOptimization, awaitPendingClose, buildSeqNoStats, close, commitStats, config, denseVectorStats, denseVectorStats, docsStats, docStats, ensureOpen, ensureOpen, externalRefresh, failEngine, fillSegmentStats, flush, flush, flush, flushAndClose, getEngineConfig, getFromSearcher, getFromTranslog, getLastUnsafeSegmentGenerationForGets, getLastWriteNanos, getMaxSeenAutoIdTimestamp, getMergeStats, getTotalFlushTimeExcludingWaitingOnLockInMillis, isClosing, isDocumentInLiveVersionMap, isDrainedForClose, maybeFailEngine, onSearcherCreation, onSettingsChanged, prepareForEngineReset, readIndexVersion, recoverFromTranslog, segmentsStats, shardFieldInfos, shardFieldStats, shardFieldStats, sparseVectorStats, sparseVectorStats, wrapForAssertions, writerSegmentStats
-
Field Details
-
FIELD_RANGE_SEARCH_SOURCE
- See Also:
-
translogStats
-
-
Constructor Details
-
ReadOnlyEngine
public ReadOnlyEngine(EngineConfig config, @Nullable SeqNoStats seqNoStats, @Nullable TranslogStats translogStats, boolean obtainLock, Function<org.apache.lucene.index.DirectoryReader, org.apache.lucene.index.DirectoryReader> readerWrapperFunction, boolean requireCompleteHistory, boolean lazilyLoadSoftDeletes) Creates a new ReadOnlyEngine. This ctor can also be used to open a read-only engine on top of an already opened read-write engine. It allows to optionally obtain the writer locks for the shard which would time-out if another engine is still open.- Parameters:
config- the engine configurationseqNoStats- sequence number statistics for this engine or null if not providedtranslogStats- translog stats for this engine or null if not providedobtainLock- iftruethis engine will try to obtain theIndexWriter.WRITE_LOCK_NAMElock. Otherwise the lock won't be obtainedreaderWrapperFunction- allows to wrap the index-reader for this engine.requireCompleteHistory- indicates whether this engine permits an incomplete history (i.e. LCP < MSN)lazilyLoadSoftDeletes- indicates whether this engine should load the soft-delete based liveDocs eagerly, or on first access
-
-
Method Details
-
ensureMaxSeqNoEqualsToGlobalCheckpoint
-
assertMaxSeqNoEqualsToGlobalCheckpoint
protected boolean assertMaxSeqNoEqualsToGlobalCheckpoint(long maxSeqNo, long globalCheckpoint) -
verifyEngineBeforeIndexClosing
Description copied from class:EnginePerforms the pre-closing checks on theEngine.- Overrides:
verifyEngineBeforeIndexClosingin classEngine- Throws:
IllegalStateException- if the sanity checks failed
-
wrapReader
protected final ElasticsearchDirectoryReader wrapReader(org.apache.lucene.index.DirectoryReader reader, Function<org.apache.lucene.index.DirectoryReader, org.apache.lucene.index.DirectoryReader> readerWrapperFunction, @Nullable ESCacheHelper esCacheHelper) throws IOException- Throws:
IOException
-
open
protected org.apache.lucene.index.DirectoryReader open(org.apache.lucene.index.IndexCommit commit) throws IOException - Throws:
IOException
-
closeNoLock
Description copied from class:EngineCloses the engine without acquiring any refs or locks. The caller should either have changedEngine.isClosingfromfalsetotrueor else must hold theEngine.failEngineLock. The implementation must decrement the supplied latch when done.- Specified by:
closeNoLockin classEngine
-
get
public Engine.GetResult get(Engine.Get get, MappingLookup mappingLookup, DocumentParser documentParser, Function<Engine.Searcher, Engine.Searcher> searcherWrapper) -
getReferenceManager
protected org.apache.lucene.search.ReferenceManager<ElasticsearchDirectoryReader> getReferenceManager(Engine.SearcherScope scope) - Specified by:
getReferenceManagerin classEngine
-
getLastCommittedSegmentInfos
public org.apache.lucene.index.SegmentInfos getLastCommittedSegmentInfos()- Specified by:
getLastCommittedSegmentInfosin classEngine
-
getHistoryUUID
Description copied from class:Enginereturns the history uuid for the engine- Specified by:
getHistoryUUIDin classEngine
-
getWritingBytes
public long getWritingBytes()Description copied from class:EngineReturns how many bytes we are currently moving from heap to disk- Specified by:
getWritingBytesin classEngine
-
getIndexThrottleTimeInMillis
public long getIndexThrottleTimeInMillis()Description copied from class:EngineReturns the number of milliseconds this engine was under index throttling.- Specified by:
getIndexThrottleTimeInMillisin classEngine
-
isThrottled
public boolean isThrottled()Description copied from class:EngineReturns thetrueiff this engine is currently under index throttling.- Specified by:
isThrottledin classEngine- See Also:
-
index
Description copied from class:EnginePerform document index operation on the engine- Specified by:
indexin classEngine- Parameters:
index- operation to perform- Returns:
Engine.IndexResultcontaining updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown
-
delete
Description copied from class:EnginePerform document delete operation on the engine- Specified by:
deletein classEngine- Parameters:
delete- operation to perform- Returns:
Engine.DeleteResultcontaining updated translog location, version and document specific failures Note: engine level failures (i.e. persistent engine failures) are thrown
-
noOp
-
isTranslogSyncNeeded
public boolean isTranslogSyncNeeded()Description copied from class:EngineChecks if the underlying storage sync is required.- Specified by:
isTranslogSyncNeededin classEngine
-
asyncEnsureTranslogSynced
Description copied from class:EngineEnsures that the location has been written to the underlying storage.- Specified by:
asyncEnsureTranslogSyncedin classEngine
-
asyncEnsureGlobalCheckpointSynced
Description copied from class:EngineEnsures that the global checkpoint has been persisted to the underlying storage.- Specified by:
asyncEnsureGlobalCheckpointSyncedin classEngine
-
syncTranslog
public void syncTranslog()- Specified by:
syncTranslogin classEngine
-
acquireHistoryRetentionLock
Description copied from class:EngineAcquires a lock on Lucene soft-deleted documents to prevent them from being trimmed- Specified by:
acquireHistoryRetentionLockin classEngine
-
countChanges
Description copied from class:EngineCounts the number of operations in the range of the given sequence numbers.- Specified by:
countChangesin classEngine- Parameters:
source- the source of the requestfromSeqNo- the start sequence number (inclusive)toSeqNo- the end sequence number (inclusive)- Throws:
IOException- See Also:
-
newChangesSnapshot
public Translog.Snapshot newChangesSnapshot(String source, long fromSeqNo, long toSeqNo, boolean requiredFullRange, boolean singleConsumer, boolean accessStats, long maxChunkSize) Description copied from class:EngineCreates a new history snapshot from Lucene for reading operations whose seqno in the requesting seqno range (both inclusive). This feature requires soft-deletes enabled. If soft-deletes are disabled, this method will throw anIllegalStateException.- Specified by:
newChangesSnapshotin classEngine
-
hasCompleteOperationHistory
Description copied from class:EngineChecks if this engine has every operations sincestartingSeqNo(inclusive) in its history (either Lucene or translog)- Specified by:
hasCompleteOperationHistoryin classEngine
-
getMinRetainedSeqNo
public long getMinRetainedSeqNo()Description copied from class:EngineGets the minimum retained sequence number for this engine.- Specified by:
getMinRetainedSeqNoin classEngine- Returns:
- the minimum retained sequence number
-
getTranslogStats
- Specified by:
getTranslogStatsin classEngine
-
getTranslogLastWriteLocation
Description copied from class:EngineReturns the last location that the translog of this engine has written into.- Specified by:
getTranslogLastWriteLocationin classEngine
-
getMaxSeqNo
public long getMaxSeqNo()- Specified by:
getMaxSeqNoin classEngine- Returns:
- the max issued or seen seqNo for this Engine
-
getProcessedLocalCheckpoint
public long getProcessedLocalCheckpoint()- Specified by:
getProcessedLocalCheckpointin classEngine- Returns:
- the processed local checkpoint for this Engine
-
getPersistedLocalCheckpoint
public long getPersistedLocalCheckpoint()- Specified by:
getPersistedLocalCheckpointin classEngine- Returns:
- the persisted local checkpoint for this Engine
-
getSeqNoStats
- Specified by:
getSeqNoStatsin classEngine- Returns:
- a
SeqNoStatsobject, using local state and the supplied global checkpoint
-
getLastSyncedGlobalCheckpoint
public long getLastSyncedGlobalCheckpoint()Description copied from class:EngineReturns the latest global checkpoint value that has been persisted in the underlying storage (i.e. translog's checkpoint)- Specified by:
getLastSyncedGlobalCheckpointin classEngine
-
getIndexBufferRAMBytesUsed
public long getIndexBufferRAMBytesUsed()Description copied from class:EngineHow much heap is used that would be freed by a refresh. This includes both the current memory being freed and any remaining memory usage that could be freed, e.g., by refreshing. Note that this may throwAlreadyClosedException.- Specified by:
getIndexBufferRAMBytesUsedin classEngine
-
segments
Description copied from class:EngineThe list of segments in the engine. -
segments
-
refresh
Description copied from class:EngineSynchronously refreshes the engine for new search operations to reflect the latest changes. -
maybeRefresh
public void maybeRefresh(String source, ActionListener<Engine.RefreshResult> listener) throws EngineException Description copied from class:EngineAsynchronously refreshes the engine for new search operations to reflect the latest changes unless another thread is already refreshing or reseting the engine concurrently.- Specified by:
maybeRefreshin classEngine- Throws:
EngineException
-
writeIndexingBuffer
public void writeIndexingBuffer()Description copied from class:EngineCalled when our engine is using too much heap and should move buffered indexed/deleted documents to disk.- Specified by:
writeIndexingBufferin classEngine
-
shouldPeriodicallyFlush
public boolean shouldPeriodicallyFlush()Description copied from class:EngineChecks if this engine should be flushed periodically. This check is mainly based on the uncommitted translog size and the translog flush threshold setting.- Specified by:
shouldPeriodicallyFlushin classEngine
-
flushHoldingLock
protected void flushHoldingLock(boolean force, boolean waitIfOngoing, ActionListener<Engine.FlushResult> listener) throws EngineException Description copied from class:EngineThe actual implementation ofEngine.flush(boolean, boolean, ActionListener), to be called either when holding a ref that ensures the engine remains open, or holdingIndexShard#engineMutexwhile closing the engine.- Specified by:
flushHoldingLockin classEngine- Throws:
EngineException
-
forceMerge
public void forceMerge(boolean flush, int maxNumSegments, boolean onlyExpungeDeletes, String forceMergeUUID) Description copied from class:EngineTriggers a forced merge on this engine- Specified by:
forceMergein classEngine
-
acquireLastIndexCommit
Description copied from class:EngineSnapshots the most recent index and returns a handle to it. If needed will try and "commit" the lucene index to make sure we have a "fresh" copy of the files to snapshot.- Specified by:
acquireLastIndexCommitin classEngine- Parameters:
flushFirst- indicates whether the engine should flush before returning the snapshot
-
acquireSafeIndexCommit
Description copied from class:EngineSnapshots the most recent safe index commit from the engine.- Specified by:
acquireSafeIndexCommitin classEngine
-
getSafeCommitInfo
- Specified by:
getSafeCommitInfoin classEngine- Returns:
- a summary of the contents of the current safe commit
-
activateThrottling
public void activateThrottling()Description copied from class:EngineRequest that this engine throttle incoming indexing requests to one thread. Must be matched by a later call toEngine.deactivateThrottling().- Specified by:
activateThrottlingin classEngine
-
deactivateThrottling
public void deactivateThrottling()Description copied from class:EngineReverses a previousEngine.activateThrottling()call.- Specified by:
deactivateThrottlingin classEngine
-
suspendThrottling
public void suspendThrottling()Description copied from class:EngineIf indexing is throttled to the point where it is paused completely, another task trying to get indexing permits might want to pause throttling by letting one thread pass at a time so that it does not get starved.- Specified by:
suspendThrottlingin classEngine
-
resumeThrottling
public void resumeThrottling()Description copied from class:EngineReverses a previousEngine.suspendThrottling()call.- Specified by:
resumeThrottlingin classEngine
-
trimUnreferencedTranslogFiles
public void trimUnreferencedTranslogFiles()Description copied from class:Enginechecks and removes translog files that no longer need to be retained. SeeTranslogDeletionPolicyfor details- Specified by:
trimUnreferencedTranslogFilesin classEngine
-
shouldRollTranslogGeneration
public boolean shouldRollTranslogGeneration()Description copied from class:EngineTests whether or not the translog generation should be rolled to a new generation. This test is based on the size of the current generation compared to the configured generation threshold size.- Specified by:
shouldRollTranslogGenerationin classEngine- Returns:
trueif the current generation should be rolled to a new generation
-
rollTranslogGeneration
public void rollTranslogGeneration()Description copied from class:EngineRolls the translog generation and cleans unneeded.- Specified by:
rollTranslogGenerationin classEngine
-
restoreLocalHistoryFromTranslog
Description copied from class:EngineThis method replays translog to restore the Lucene index which might be reverted previously. This ensures that all acknowledged writes are restored correctly when this engine is promoted.- Specified by:
restoreLocalHistoryFromTranslogin classEngine- Returns:
- the number of translog operations have been recovered
-
fillSeqNoGaps
public int fillSeqNoGaps(long primaryTerm) Description copied from class:EngineFills up the local checkpoints history with no-ops until the local checkpoint and the max seen sequence ID are identical.- Specified by:
fillSeqNoGapsin classEngine- Parameters:
primaryTerm- the shards primary term this engine was created for- Returns:
- the number of no-ops added
-
recoverFromTranslog
public void recoverFromTranslog(Engine.TranslogRecoveryRunner translogRecoveryRunner, long recoverUpToSeqNo, ActionListener<Void> listener) Description copied from class:EnginePerforms recovery from the transaction log up torecoverUpToSeqNo(inclusive). This operation will close the engine if the recovery fails.- Specified by:
recoverFromTranslogin classEngine- Parameters:
translogRecoveryRunner- the translog recovery runnerrecoverUpToSeqNo- the upper bound, inclusive, of sequence number to be recoveredlistener- listener notified on completion of the recovery, whether successful or otherwise
-
skipTranslogRecovery
public void skipTranslogRecovery()Description copied from class:EngineDo not replay translog operations, but make the engine be ready.- Specified by:
skipTranslogRecoveryin classEngine
-
trimOperationsFromTranslog
public void trimOperationsFromTranslog(long belowTerm, long aboveSeqNo) Description copied from class:EngineTrims translog for terms belowbelowTermand seq# aboveaboveSeqNo- Specified by:
trimOperationsFromTranslogin classEngine- See Also:
-
maybePruneDeletes
public void maybePruneDeletes()Description copied from class:EngineTries to prune buffered deletes from the version map.- Specified by:
maybePruneDeletesin classEngine
-
updateMaxUnsafeAutoIdTimestamp
public void updateMaxUnsafeAutoIdTimestamp(long newTimestamp) Description copied from class:EngineForces this engine to advance its max_unsafe_auto_id_timestamp marker to at least the given timestamp. The engine will disable optimization for all append-only whose timestamp at mostnewTimestamp.- Specified by:
updateMaxUnsafeAutoIdTimestampin classEngine
-
refreshNeeded
public boolean refreshNeeded()- Overrides:
refreshNeededin classEngine
-
getMaxSeqNoOfUpdatesOrDeletes
public long getMaxSeqNoOfUpdatesOrDeletes()Description copied from class:EngineReturns the maximum sequence number of either update or delete operations have been processed in this engine or the sequence number fromEngine.advanceMaxSeqNoOfUpdatesOrDeletes(long). An index request is considered as an update operation if it overwrites the existing documents in Lucene index with the same document id.A note on the optimization using max_seq_no_of_updates_or_deletes: For each operation O, the key invariants are:
- I1: There is no operation on docID(O) with seqno that is > MSU(O) and < seqno(O)
- I2: If MSU(O) < seqno(O) then docID(O) did not exist when O was applied; more precisely, if there is any O' with seqno(O') < seqno(O) and docID(O') = docID(O) then the one with the greatest seqno is a delete.
When a receiving shard (either a replica or a follower) receives an operation O, it must first ensure its own MSU at least MSU(O), and then compares its MSU to its local checkpoint (LCP). If LCP < MSU then there's a gap: there may be some operations that act on docID(O) about which we do not yet know, so we cannot perform an add. Note this also covers the case where a future operation O' with seqNo(O') > seqNo(O) and docId(O') = docID(O) is processed before O. In that case MSU(O') is at least seqno(O') and this means MSU >= seqNo(O') > seqNo(O) > LCP (because O wasn't processed yet).
However, if MSU <= LCP then there is no gap: we have processed every operation <= LCP, and no operation O' with seqno(O') > LCP and seqno(O') < seqno(O) also has docID(O') = docID(O), because such an operation would have seqno(O') > LCP >= MSU >= MSU(O) which contradicts the first invariant. Furthermore in this case we immediately know that docID(O) has been deleted (or never existed) without needing to check Lucene for the following reason. If there's no earlier operation on docID(O) then this is clear, so suppose instead that the preceding operation on docID(O) is O': 1. The first invariant above tells us that seqno(O') <= MSU(O) <= LCP so we have already applied O' to Lucene. 2. Also MSU(O) <= MSU <= LCP < seqno(O) (we discard O if seqno(O) <= LCP) so the second invariant applies, meaning that the O' was a delete.
Therefore, if MSU <= LCP < seqno(O) we know that O can safely be optimized with and added to lucene with addDocument. Moreover, operations that are optimized using the MSU optimization must not be processed twice as this will create duplicates in Lucene. To avoid this we check the local checkpoint tracker to see if an operation was already processed.
- Specified by:
getMaxSeqNoOfUpdatesOrDeletesin classEngine- See Also:
-
advanceMaxSeqNoOfUpdatesOrDeletes
public void advanceMaxSeqNoOfUpdatesOrDeletes(long maxSeqNoOfUpdatesOnPrimary) Description copied from class:EngineA replica shard receives a new max_seq_no_of_updates from its primary shard, then calls this method to advance this marker to at least the given sequence number.- Specified by:
advanceMaxSeqNoOfUpdatesOrDeletesin classEngine
-
openDirectory
protected org.apache.lucene.index.DirectoryReader openDirectory(org.apache.lucene.store.Directory directory) throws IOException - Throws:
IOException
-
completionStats
Description copied from class:EngineReturns theCompletionStatsfor this engine- Specified by:
completionStatsin classEngine
-
getRawFieldRange
- Specified by:
getRawFieldRangein classEngine- Returns:
- a
ShardLongFieldRangecontaining the min and max raw values of the given field for this shard, orShardLongFieldRange.EMPTYif this field is not found or empty. - Throws:
IOException
-
acquireSearcherSupplier
public Engine.SearcherSupplier acquireSearcherSupplier(Function<Engine.Searcher, Engine.Searcher> wrapper, Engine.SearcherScope scope) throws EngineExceptionDescription copied from class:EngineAcquires a point-in-time reader that can be used to createEngine.Searchers on demand.- Overrides:
acquireSearcherSupplierin classEngine- Throws:
EngineException
-
getCommitId
-