Module org.elasticsearch.server
Package org.elasticsearch.index.engine
Class ThreadPoolMergeScheduler
java.lang.Object
org.apache.lucene.index.MergeScheduler
org.elasticsearch.index.engine.ThreadPoolMergeScheduler
- All Implemented Interfaces:
Closeable,AutoCloseable,ElasticsearchMergeScheduler
public class ThreadPoolMergeScheduler
extends org.apache.lucene.index.MergeScheduler
implements ElasticsearchMergeScheduler
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.lucene.index.MergeScheduler
org.apache.lucene.index.MergeScheduler.MergeSource -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final org.apache.logging.log4j.LoggerThis setting switches between the originalElasticsearchConcurrentMergeSchedulerand the newThreadPoolMergeSchedulermerge scheduler implementations (the latter is switched ON by default).Fields inherited from class org.apache.lucene.index.MergeScheduler
infoStream -
Constructor Summary
ConstructorsConstructorDescriptionThreadPoolMergeScheduler(ShardId shardId, IndexSettings indexSettings, ThreadPoolMergeExecutorService threadPoolMergeExecutorService, MergeMemoryEstimateProvider mergeMemoryEstimateProvider, MergeMetrics mergeMetrics) Creates a thread-pool-based merge scheduler that runs merges in a thread pool. -
Method Summary
Modifier and TypeMethodDescriptionprotected voidafterMerge(OnGoingMerge merge) A callback allowing for custom logic after an actual merge starts.protected voidbeforeMerge(OnGoingMerge merge) A callback allowing for custom logic before an actual merge starts.org.apache.lucene.index.MergeSchedulerclone()voidclose()protected voiddisableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount) A callback that's invoked when indexing should un-throttle because merging caught up.protected voidenableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount) A callback that's invoked when indexing should throttle down indexing in order to let merging to catch up.protected intReturns the maximum number of active merges before being throttledprotected intReturns the maximum number of threads running merges before being throttledorg.apache.lucene.index.MergeSchedulerprotected voidA callback for exceptions thrown while merging.protected booleanReturns true if IO-throttling is enabledvoidmerge(org.apache.lucene.index.MergeScheduler.MergeSource mergeSource, org.apache.lucene.index.MergeTrigger trigger) protected voidA callback allowing for custom logic after a merge is executed or aborted.protected voidmergeQueued(OnGoingMerge merge) A callback allowing for custom logic when a merge is queued.protected voidvoidprotected booleanReturns true if scheduled merges should be skipped (aborted)stats()protected booleanverbose()org.apache.lucene.store.DirectorywrapForMerge(org.apache.lucene.index.MergePolicy.OneMerge merge, org.apache.lucene.store.Directory in) Methods inherited from class org.apache.lucene.index.MergeScheduler
getIntraMergeExecutor
-
Field Details
-
USE_THREAD_POOL_MERGE_SCHEDULER_SETTING
This setting switches between the originalElasticsearchConcurrentMergeSchedulerand the newThreadPoolMergeSchedulermerge scheduler implementations (the latter is switched ON by default). This setting is purposefully undocumented, because we expect that only the newThreadPoolMergeSchedulerimplementation (which is enabled by default) be used from now on. Our users should not touch this setting in their deployments, unless consulting with engineering, because the original implementation should only be used (by setting this tofalse) to get around unexpected issues with the new one. The setting is also deprecated in the hope that any unexpected issues with the new merge scheduler implementation are promptly resolved, such that, in the near future, there's never a need to switch to the original implementation, which will then be removed together with this setting. -
logger
protected final org.apache.logging.log4j.Logger logger
-
-
Constructor Details
-
ThreadPoolMergeScheduler
public ThreadPoolMergeScheduler(ShardId shardId, IndexSettings indexSettings, ThreadPoolMergeExecutorService threadPoolMergeExecutorService, MergeMemoryEstimateProvider mergeMemoryEstimateProvider, MergeMetrics mergeMetrics) Creates a thread-pool-based merge scheduler that runs merges in a thread pool.- Parameters:
shardId- the shard id associated with this merge schedulerindexSettings- used to obtain theMergeSchedulerConfigthreadPoolMergeExecutorService- the executor service used to execute merge tasks from this schedulermergeMemoryEstimateProvider- provides an estimate for how much memory a merge will takemergeMetrics- metrics related to merges
-
-
Method Details
-
onGoingMerges
- Specified by:
onGoingMergesin interfaceElasticsearchMergeScheduler
-
stats
- Specified by:
statsin interfaceElasticsearchMergeScheduler
-
getMergeScheduler
public org.apache.lucene.index.MergeScheduler getMergeScheduler()- Specified by:
getMergeSchedulerin interfaceElasticsearchMergeScheduler
-
refreshConfig
public void refreshConfig()- Specified by:
refreshConfigin interfaceElasticsearchMergeScheduler
-
merge
public void merge(org.apache.lucene.index.MergeScheduler.MergeSource mergeSource, org.apache.lucene.index.MergeTrigger trigger) - Specified by:
mergein classorg.apache.lucene.index.MergeScheduler
-
clone
public org.apache.lucene.index.MergeScheduler clone() -
beforeMerge
A callback allowing for custom logic before an actual merge starts. -
afterMerge
A callback allowing for custom logic after an actual merge starts. -
mergeQueued
A callback allowing for custom logic when a merge is queued. -
mergeExecutedOrAborted
A callback allowing for custom logic after a merge is executed or aborted. -
enableIndexingThrottling
protected void enableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount) A callback that's invoked when indexing should throttle down indexing in order to let merging to catch up. -
disableIndexingThrottling
protected void disableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount) A callback that's invoked when indexing should un-throttle because merging caught up. This is invoked sometime afterenableIndexingThrottling(int, int, int)was invoked in the first place. -
shouldSkipMerge
protected boolean shouldSkipMerge()Returns true if scheduled merges should be skipped (aborted) -
isAutoThrottle
protected boolean isAutoThrottle()Returns true if IO-throttling is enabled -
getMaxMergeCount
protected int getMaxMergeCount()Returns the maximum number of active merges before being throttled -
getMaxThreadCount
protected int getMaxThreadCount()Returns the maximum number of threads running merges before being throttled -
handleMergeException
A callback for exceptions thrown while merging. -
wrapForMerge
public org.apache.lucene.store.Directory wrapForMerge(org.apache.lucene.index.MergePolicy.OneMerge merge, org.apache.lucene.store.Directory in) - Overrides:
wrapForMergein classorg.apache.lucene.index.MergeScheduler
-
verbose
protected boolean verbose()- Overrides:
verbosein classorg.apache.lucene.index.MergeScheduler
-
message
- Overrides:
messagein classorg.apache.lucene.index.MergeScheduler
-
close
- Specified by:
closein interfaceAutoCloseable- Specified by:
closein interfaceCloseable- Overrides:
closein classorg.apache.lucene.index.MergeScheduler- Throws:
IOException
-