Class ThreadPoolMergeScheduler

java.lang.Object
org.apache.lucene.index.MergeScheduler
org.elasticsearch.index.engine.ThreadPoolMergeScheduler
All Implemented Interfaces:
Closeable, AutoCloseable, ElasticsearchMergeScheduler

public class ThreadPoolMergeScheduler extends org.apache.lucene.index.MergeScheduler implements ElasticsearchMergeScheduler
  • Field Details

    • USE_THREAD_POOL_MERGE_SCHEDULER_SETTING

      public static final Setting<Boolean> USE_THREAD_POOL_MERGE_SCHEDULER_SETTING
      This setting switches between the original ElasticsearchConcurrentMergeScheduler and the new ThreadPoolMergeScheduler merge scheduler implementations (the latter is switched ON by default). This setting is purposefully undocumented, because we expect that only the new ThreadPoolMergeScheduler implementation (which is enabled by default) be used from now on. Our users should not touch this setting in their deployments, unless consulting with engineering, because the original implementation should only be used (by setting this to false) to get around unexpected issues with the new one. The setting is also deprecated in the hope that any unexpected issues with the new merge scheduler implementation are promptly resolved, such that, in the near future, there's never a need to switch to the original implementation, which will then be removed together with this setting.
    • logger

      protected final org.apache.logging.log4j.Logger logger
  • Constructor Details

    • ThreadPoolMergeScheduler

      public ThreadPoolMergeScheduler(ShardId shardId, IndexSettings indexSettings, ThreadPoolMergeExecutorService threadPoolMergeExecutorService, MergeMemoryEstimateProvider mergeMemoryEstimateProvider)
      Creates a thread-pool-based merge scheduler that runs merges in a thread pool.
      Parameters:
      shardId - the shard id associated with this merge scheduler
      indexSettings - used to obtain the MergeSchedulerConfig
      threadPoolMergeExecutorService - the executor service used to execute merge tasks from this scheduler
      mergeMemoryEstimateProvider - provides an estimate for how much memory a merge will take
  • Method Details

    • onGoingMerges

      public Set<OnGoingMerge> onGoingMerges()
      Specified by:
      onGoingMerges in interface ElasticsearchMergeScheduler
    • stats

      public MergeStats stats()
      Specified by:
      stats in interface ElasticsearchMergeScheduler
    • getMergeScheduler

      public org.apache.lucene.index.MergeScheduler getMergeScheduler()
      Specified by:
      getMergeScheduler in interface ElasticsearchMergeScheduler
    • refreshConfig

      public void refreshConfig()
      Specified by:
      refreshConfig in interface ElasticsearchMergeScheduler
    • merge

      public void merge(org.apache.lucene.index.MergeScheduler.MergeSource mergeSource, org.apache.lucene.index.MergeTrigger trigger)
      Specified by:
      merge in class org.apache.lucene.index.MergeScheduler
    • clone

      public org.apache.lucene.index.MergeScheduler clone()
      Overrides:
      clone in class Object
    • beforeMerge

      protected void beforeMerge(OnGoingMerge merge)
      A callback allowing for custom logic before an actual merge starts.
    • afterMerge

      protected void afterMerge(OnGoingMerge merge)
      A callback allowing for custom logic after an actual merge starts.
    • mergeQueued

      protected void mergeQueued(OnGoingMerge merge)
      A callback allowing for custom logic when a merge is queued.
    • mergeExecutedOrAborted

      protected void mergeExecutedOrAborted(OnGoingMerge merge)
      A callback allowing for custom logic after a merge is executed or aborted.
    • enableIndexingThrottling

      protected void enableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount)
      A callback that's invoked when indexing should throttle down indexing in order to let merging to catch up.
    • disableIndexingThrottling

      protected void disableIndexingThrottling(int numRunningMerges, int numQueuedMerges, int configuredMaxMergeCount)
      A callback that's invoked when indexing should un-throttle because merging caught up. This is invoked sometime after enableIndexingThrottling(int, int, int) was invoked in the first place.
    • shouldSkipMerge

      protected boolean shouldSkipMerge()
      Returns true if scheduled merges should be skipped (aborted)
    • isAutoThrottle

      protected boolean isAutoThrottle()
      Returns true if IO-throttling is enabled
    • getMaxMergeCount

      protected int getMaxMergeCount()
      Returns the maximum number of active merges before being throttled
    • getMaxThreadCount

      protected int getMaxThreadCount()
      Returns the maximum number of threads running merges before being throttled
    • handleMergeException

      protected void handleMergeException(Throwable t)
      A callback for exceptions thrown while merging.
    • wrapForMerge

      public org.apache.lucene.store.Directory wrapForMerge(org.apache.lucene.index.MergePolicy.OneMerge merge, org.apache.lucene.store.Directory in)
      Overrides:
      wrapForMerge in class org.apache.lucene.index.MergeScheduler
    • verbose

      protected boolean verbose()
      Overrides:
      verbose in class org.apache.lucene.index.MergeScheduler
    • message

      protected void message(String message)
      Overrides:
      message in class org.apache.lucene.index.MergeScheduler
    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class org.apache.lucene.index.MergeScheduler
      Throws:
      IOException