Package org.elasticsearch.xpack.esql.plugin

Class QueryPragmas

java.lang.Object
org.elasticsearch.xpack.esql.plugin.QueryPragmas
All Implemented Interfaces:
Writeable
public final class QueryPragmas extends Object implements Writeable
Holds the pragmas for an ESQL query. Just a wrapper of settings for now.

  • Field Details

    • EXCHANGE_BUFFER_SIZE

      public static final Setting<Integer> EXCHANGE_BUFFER_SIZE

    • ENRICH_MAX_WORKERS

      public static final Setting<Integer> ENRICH_MAX_WORKERS

    • TASK_CONCURRENCY

      public static final Setting<Integer> TASK_CONCURRENCY

    • DATA_PARTITIONING

      public static final Setting<String> DATA_PARTITIONING
      How to cut slices to cut each shard into. Is parsed to the enum DataPartitioning which has more documentation. Not an Setting.enumSetting(java.lang.Class<T>, java.lang.String, T, org.elasticsearch.common.settings.Setting.Property...) because those can't have null defaults. null here means "use the default from the cluster setting named PlannerSettings.DEFAULT_DATA_PARTITIONING."

    • PAGE_SIZE

      public static final Setting<Integer> PAGE_SIZE
      Size of a page in entries with 0 being a special value asking to adaptively size based on the number of columns in the page.

    • STATUS_INTERVAL

      public static final Setting<TimeValue> STATUS_INTERVAL
      The minimum interval between syncs of the DriverStatus, making the status available to task API.

    • MAX_CONCURRENT_NODES_PER_CLUSTER

      public static final Setting<Integer> MAX_CONCURRENT_NODES_PER_CLUSTER

    • MAX_CONCURRENT_SHARDS_PER_NODE

      public static final Setting<Integer> MAX_CONCURRENT_SHARDS_PER_NODE

    • UNAVAILABLE_SHARD_RESOLUTION_ATTEMPTS

      public static final Setting<Integer> UNAVAILABLE_SHARD_RESOLUTION_ATTEMPTS

    • NODE_LEVEL_REDUCTION

      public static final Setting<Boolean> NODE_LEVEL_REDUCTION

    • FOLD_LIMIT

      public static final Setting<ByteSizeValue> FOLD_LIMIT

    • FIELD_EXTRACT_PREFERENCE

      public static final Setting<MappedFieldType.FieldExtractPreference> FIELD_EXTRACT_PREFERENCE

    • ROUNDTO_PUSHDOWN_THRESHOLD

      public static final Setting<Integer> ROUNDTO_PUSHDOWN_THRESHOLD
      The maximum number of rounding points to push down to Lucene for the roundTo function at query level. ReplaceRoundToWithQueryAndTags checks this threshold before rewriting RoundTo to range queries. There is also a cluster level ESQL_ROUNDTO_PUSHDOWN_THRESHOLD defined in EsqlFlags. The query level threshold defaults to -1, which means this query level setting is not set and cluster level upper limit will be used. The cluster level threshold defaults to 127, it is the same as the maximum number of buckets used in Rounding. If query level threshold is set to greater than or equals to 0, the query level threshold will be used, and it overrides the cluster level threshold. If the query level threshold is set to 0, no RoundTo pushdown will be performed.

    • MAX_KEYWORD_SORT_FIELDS

      public static final Setting<Integer> MAX_KEYWORD_SORT_FIELDS
      Query-level override for the maximum number of keyword sort fields allowed when pushing TopN to Lucene. Defaults to -1, meaning the cluster-level setting PlannerSettings.MAX_KEYWORD_SORT_FIELDS is used. When set to a value >= 0, it overrides the cluster-level threshold for this query only. The resolution logic lives in PushTopNToSource.

    • EXTERNAL_DISTRIBUTION

      public static final Setting<String> EXTERNAL_DISTRIBUTION
      Controls how external source queries are distributed across nodes. Valid values: "adaptive" (default), "coordinator_only", "round_robin".

    • IN_SUBQUERY_HASH_JOIN_THRESHOLD

      public static final Setting<Integer> IN_SUBQUERY_HASH_JOIN_THRESHOLD
      Query-level override for the IN subquery hash join threshold. Defaults to -1, meaning the cluster-level setting PlannerSettings.IN_SUBQUERY_HASH_JOIN_THRESHOLD is used. When set to a value >= 0, it overrides the cluster-level threshold for this query only.

    • BRANCH_PARALLEL_DEGREE

      public static final Setting<Integer> BRANCH_PARALLEL_DEGREE
      The number of branches to execute in parallel. This is a safeguard to avoid overloading the cluster with too many parallel branches. This applies to forks and subqueries.

    • PARSING_PARALLELISM

      public static final Setting<Integer> PARSING_PARALLELISM
      Number of parallel parser threads for intra-file text format parsing (CSV, NDJSON). Defaults to allocated processors. Set to 1 to disable parallel parsing.

    • MAX_CONCURRENT_OPEN_SEGMENTS

      public static final Setting<Integer> MAX_CONCURRENT_OPEN_SEGMENTS
      Per-file cap on the number of intra-file byte-range segments whose object-store read streams are open at once during parallel text parsing. Each open segment holds a storage read stream (one S3 GetObject) plus, for buffering readers like NDJSON, a per-segment byte[]; the consumer emits a segment's pages as soon as that segment finishes parsing (completion order, not strict segment order), so this is a shallow read-ahead width, not a parallelism. It is deliberately not parsing_parallelism: a file is already split into about parsing_parallelism segments and many files read concurrently, so aligning this with the thread count would fan a wide multi-file glob into far too many concurrent object-store reads. A small default bounds that fan-out independent of file count/length. Safeguard in the spirit of BRANCH_PARALLEL_DEGREE.

      This is a per-file cap. The node-wide bound on concurrently-open segment streams is roughly (data-node driver instances) × max_concurrent_open_segments × (files open per driver) — tune with that product in mind, not this value alone. The default is sourced from SourceOperatorContext.DEFAULT_MAX_CONCURRENT_OPEN_SEGMENTS.

    • MAX_RECORD_SIZE

      public static final Setting<ByteSizeValue> MAX_RECORD_SIZE
      Bytes the streaming external-source parser may buffer for a single record before failing the query — guards against a scanner that never finds a boundary reading the input without bound. Defaults to SegmentableFormatReader.DEFAULT_MAX_RECORD_BYTES. Bounded to [1, Integer.MAX_VALUE] bytes: the cap is held as an int downstream, so an out-of-range value is rejected at parse time rather than overflowing later.

    • FORCE_DOC_SEQUENCE

      public static final Setting<Boolean> FORCE_DOC_SEQUENCE
      When true, forces all non-single-segment pages through ValuesFromDocSequence regardless of the number of BYTES_REF fields. Intended for testing the correctness of doc-sequence loading.

    • MIN_DOCS_PER_SLICE

      public static final Setting<Integer> MIN_DOCS_PER_SLICE
      Query-level override for the minimum number of docs per Lucene slice (see LuceneSliceQueue.MIN_DOCS_PER_SLICE). Defaults to -1, meaning the compute-engine default floor is used. When set to a value > 0, it overrides that floor for this query only.

      Primarily a testing lever: the default floor collapses small indices to a single slice, which disables DOC/SEGMENT parallelism in tests that index only a few documents. Lowering it lets such tests exercise the multi-slice partitioning paths.

    • EMPTY

      public static final QueryPragmas EMPTY

    • VALID_PRAGMA_NAMES

      public static final List<String> VALID_PRAGMA_NAMES

  • Constructor Details

  • Method Details

    • settings

      public Settings settings()
      Returns the underlying settings.

    • writeTo

      public void writeTo(StreamOutput out) throws IOException
      Specified by:
      writeTo in interface Writeable
      Throws:
      IOException

    • getSettings

      public Settings getSettings()

    • exchangeBufferSize

      public int exchangeBufferSize()

    • concurrentExchangeClients

      public int concurrentExchangeClients()

    • dataPartitioning

      public DataPartitioning dataPartitioning(DataPartitioning defaultDataPartitioning)

    • taskConcurrency

      public int taskConcurrency()

    • pageSize

      public int pageSize()
      Size of a page in entries with 0 being a special value asking to adaptively size based on the number of columns in the page.

    • statusInterval

      public TimeValue statusInterval()
      The minimum interval between syncs of the DriverStatus, making the status available to task API.

    • enrichMaxWorkers

      public int enrichMaxWorkers()
      Returns the maximum number of workers for enrich lookup. A higher number of workers reduces latency but increases cluster load. Defaults to 1.

    • maxConcurrentNodesPerCluster

      public int maxConcurrentNodesPerCluster()
      The maximum number of nodes to be queried at once by this query. This is safeguard to avoid overloading the cluster.

    • maxConcurrentShardsPerNode

      public int maxConcurrentShardsPerNode()
      The maximum number of shards can be executed concurrently on a single node by this query. This is a safeguard to avoid opening and holding many shards (equivalent to many file descriptors) or having too many field infos created by a single query.

    • unavailableShardResolutionAttempts

      public int unavailableShardResolutionAttempts()
      Amount of attempts moved shards could be retried. This setting is protecting query from endlessly chasing moving shards.

    • nodeLevelReduction

      public boolean nodeLevelReduction()
      Returns true if each data node should perform a local reduction for sort, limit, topN, stats or false if the coordinator node will perform the reduction.

    • foldLimit

      public ByteSizeValue foldLimit()
      The maximum amount of memory we can use for Expression.fold(org.elasticsearch.xpack.esql.core.expression.FoldContext) during planing. This defaults to 5% of memory available on the current node. If this method is called on the coordinating node, this is 5% of the coordinating node's memory. If it's called on a data node, it's 5% of the data node. That's an exciting inconsistency. But it's important. Bigger nodes have more space to do folding.

    • fieldExtractPreference

      public MappedFieldType.FieldExtractPreference fieldExtractPreference()
      The default preference for extracting fields, defaults to NONE. Some fields must be extracted in some special way because of how they are used in the plan. But most fields can be loaded in many ways so they pick the ways that they think are most efficient in their MappedFieldType.blockLoader(org.elasticsearch.index.mapper.MappedFieldType.BlockLoaderContext) method. This can influence their choice.

    • roundToPushDownThreshold

      public int roundToPushDownThreshold()

    • maxKeywordSortFields

      public int maxKeywordSortFields()

    • externalDistribution

      public String externalDistribution()

    • parsingParallelism

      public int parsingParallelism()

    • maxConcurrentOpenSegments

      public int maxConcurrentOpenSegments()

    • maxRecordSize

      public ByteSizeValue maxRecordSize()

    • branchParallelDegree

      public int branchParallelDegree()

    • docSequenceBytesRefFieldThreshold

      public int docSequenceBytesRefFieldThreshold(int clusterDefault)
      Returns the effective doc-sequence threshold. When FORCE_DOC_SEQUENCE is true, returns 0 so that all non-single-segment pages use ValuesFromDocSequence; otherwise returns clusterDefault.

    • partialAggregationEmitKeysThreshold

      public int partialAggregationEmitKeysThreshold(int defaultThreshold)

    • partialAggregationEmitUniquenessThreshold

      public double partialAggregationEmitUniquenessThreshold(double defaultThreshold)

    • timeSeriesTargetChunkRows

      public int timeSeriesTargetChunkRows(int defaultChunkRows)

    • docsThresholdForAutoPartitioning

      public int docsThresholdForAutoPartitioning(int defaultThreshold)

    • minDocsPerSlice

      public int minDocsPerSlice(int defaultMinDocsPerSlice)
      Returns the effective minimum number of docs per Lucene slice. When MIN_DOCS_PER_SLICE is set to a positive value it overrides defaultMinDocsPerSlice; otherwise the default floor is used.

    • runtimeLexicalSearch

      public boolean runtimeLexicalSearch()

    • isEmpty

      public boolean isEmpty()

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object