Package org.elasticsearch.xpack.esql.plugin

Class EsqlPlugin

java.lang.Object

org.elasticsearch.plugins.Plugin

org.elasticsearch.xpack.esql.plugin.EsqlPlugin

All Implemented Interfaces:

Closeable, AutoCloseable, ActionPlugin, ExtensiblePlugin, SearchPlugin

public class EsqlPlugin
extends Plugin
implements ActionPlugin, ExtensiblePlugin, SearchPlugin

Nested Class Summary

Nested classes/interfaces inherited from class org.elasticsearch.plugins.Plugin

Plugin.PluginServices

Nested classes/interfaces inherited from interface org.elasticsearch.plugins.ActionPlugin

ActionPlugin.ActionHandler, ActionPlugin.RestHandlersServices

Nested classes/interfaces inherited from interface org.elasticsearch.plugins.ExtensiblePlugin

ExtensiblePlugin.ExtensionLoader

Nested classes/interfaces inherited from interface org.elasticsearch.plugins.SearchPlugin

SearchPlugin.AggregationSpec, SearchPlugin.FetchPhaseConstructionContext, SearchPlugin.GenericNamedWriteableSpec, SearchPlugin.PipelineAggregationSpec, SearchPlugin.QuerySpec<T extends QueryBuilder>, SearchPlugin.QueryVectorBuilderSpec<T extends QueryVectorBuilder>, SearchPlugin.RescorerSpec<T extends RescorerBuilder<T>>, SearchPlugin.RetrieverSpec<RB extends RetrieverBuilder>, SearchPlugin.ScoreFunctionSpec<T extends ScoreFunctionBuilder<T>>, SearchPlugin.SearchExtensionSpec<W extends NamedWriteable,P>, SearchPlugin.SearchExtSpec<T extends SearchExtBuilder>, SearchPlugin.SignificanceHeuristicSpec<T extends SignificanceHeuristic>, SearchPlugin.SuggesterSpec<T extends SuggestionBuilder<T>>

Field Summary

Fields

Modifier and Type	Field	Description
static final Setting<Boolean>	ESQL_QUERYLOG_INCLUDE_USER_SETTING
static final Setting<TimeValue>	ESQL_QUERYLOG_THRESHOLD_DEBUG_SETTING
static final Setting<TimeValue>	ESQL_QUERYLOG_THRESHOLD_INFO_SETTING
static final Setting<TimeValue>	ESQL_QUERYLOG_THRESHOLD_TRACE_SETTING
static final Setting<TimeValue>	ESQL_QUERYLOG_THRESHOLD_WARN_SETTING
static final String	ESQL_WORKER_THREAD_POOL_NAME
static final Setting<Integer>	ESQL_WORKER_THREAD_POOL_SIZE
static final String	EXTERNAL_IO_THREAD_POOL_NAME	Name of the dedicated scaling pool for blocking external blob-store I/O and the streaming parse pipeline.
static final Setting<Boolean>	FLATTENED_ENABLED	Kill switch for the flattened data type in ES|QL.
static final Setting<TimeValue>	GROK_WATCHDOG_MAX_EXECUTION_TIME	Maximum time (in milliseconds) that a GROK matcher is allowed to run before being interrupted.
static final Setting<Boolean>	LOOKUP_JOIN_STREAMING	Controls whether LOOKUP JOIN uses the streaming lookup operator, which streams pages to the lookup node instead of performing a request per input page.
static final Setting<Boolean>	QUERY_ALLOW_PARTIAL_RESULTS
static final Setting<Double>	STORED_FIELDS_SEQUENTIAL_PROPORTION	Tuning parameter for deciding when to use the "merge" stored field loader.

Constructor Summary

Constructors

Constructor	Description
EsqlPlugin()

Method Summary

Modifier and Type	Method	Description
protected BlockFactoryProvider	blockFactoryProvider(BlockFactoryBuilder builder)
void	close()
static String	computePool()	Name of the thread pool backing ES|QL compute — driver execution and the parallel worker fan-out.
Collection<?>	createComponents(Plugin.PluginServices services)
static String	externalBlobStorePool()	Name of the dedicated thread pool backing all external blob-store access: metadata discovery (glob expansion, footer reads, and schema reconciliation performed by ExternalSourceResolver) as well as the blocking data reads and the streaming parse pipeline (segmentator + parser tasks of StreamingParallelParsingCoordinator) routed through OperatorFactoryRegistry#fileReadExecutor.
List<ActionPlugin.ActionHandler>	getActions()
List<ExecutorBuilder<?>>	getExecutorBuilders(Settings settings)
List<SearchPlugin.GenericNamedWriteableSpec>	getGenericNamedWriteables()
protected XPackLicenseState	getLicenseState()
List<NamedWriteableRegistry.Entry>	getNamedWriteables()
List<NamedXContentRegistry.Entry>	getNamedXContent()
List<RestHandler>	getRestHandlers(ActionPlugin.RestHandlersServices restHandlersServices, Supplier<DiscoveryNodes> nodesInCluster, Predicate<NodeFeature> clusterSupportsFeature)
List<Setting<?>>	getSettings()	The settings defined by the ESQL plugin.
void	loadExtensions(ExtensiblePlugin.ExtensionLoader loader)

Methods inherited from class org.elasticsearch.plugins.Plugin

additionalSettings, getAdditionalIndexSettingProviders, getBootstrapChecks, getIndexTemplateMetadataUpgrader, getProjectCustomMetadataUpgraders, getSettingsFilter, onIndexModule

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.elasticsearch.plugins.ActionPlugin

getActionFilters, getMappedActionFilters, getRestHeaders, getTaskHeaders, indicesAliasesRequestValidators, mappingRequestValidators

Methods inherited from interface org.elasticsearch.plugins.SearchPlugin

getAggregationExtentions, getAggregations, getFetchSubPhases, getHighlighters, getPipelineAggregations, getQueries, getQueryVectorBuilders, getRequestCacheKeyDifferentiator, getRescorers, getRetrievers, getScoreFunctions, getSearchExts, getSignificanceHeuristics, getSuggesters

Field Details

ESQL_WORKER_THREAD_POOL_NAME

public static final String ESQL_WORKER_THREAD_POOL_NAME

See Also:

• Constant Field Values

EXTERNAL_IO_THREAD_POOL_NAME

public static final String EXTERNAL_IO_THREAD_POOL_NAME

Name of the dedicated scaling pool for blocking external blob-store I/O and the streaming parse pipeline. Sized 0..ExternalSourceSettings.externalIoThreads(Settings) — tracking the same single CPU-scaled knob (snapshot_meta shape, capped at 100, or the esql.external.max_concurrent_requests operator override) that sizes the permit semaphore and the S3/Azure SDK connection pools, so the pool cannot diverge from the concurrency the reads are permitted. Scales from 0 so idle nodes pay nothing. See externalBlobStorePool() for why this is separate from esql_worker.

See Also:

• Constant Field Values

ESQL_WORKER_THREAD_POOL_SIZE

public static final Setting<Integer> ESQL_WORKER_THREAD_POOL_SIZE

QUERY_ALLOW_PARTIAL_RESULTS

public static final Setting<Boolean> QUERY_ALLOW_PARTIAL_RESULTS

LOOKUP_JOIN_STREAMING

public static final Setting<Boolean> LOOKUP_JOIN_STREAMING

Controls whether LOOKUP JOIN uses the streaming lookup operator, which streams pages to the lookup node instead of performing a request per input page. Acts as an escape hatch to fall back to the non-streaming operator; even when enabled, streaming is only used if all target nodes support the streaming protocol.

FLATTENED_ENABLED

public static final Setting<Boolean> FLATTENED_ENABLED

Kill switch for the flattened data type in ES|QL. When false, flattened fields resolve as unsupported during index resolution (the exact pre-flattened-support behavior: FROM still works and the column is returned as unsupported, but any explicit use errors). Because field_extract only operates on flattened fields, disabling the type also disables that function transitively.

ESQL_QUERYLOG_THRESHOLD_WARN_SETTING

public static final Setting<TimeValue> ESQL_QUERYLOG_THRESHOLD_WARN_SETTING

ESQL_QUERYLOG_THRESHOLD_INFO_SETTING

public static final Setting<TimeValue> ESQL_QUERYLOG_THRESHOLD_INFO_SETTING

ESQL_QUERYLOG_THRESHOLD_DEBUG_SETTING

public static final Setting<TimeValue> ESQL_QUERYLOG_THRESHOLD_DEBUG_SETTING

ESQL_QUERYLOG_THRESHOLD_TRACE_SETTING

public static final Setting<TimeValue> ESQL_QUERYLOG_THRESHOLD_TRACE_SETTING

ESQL_QUERYLOG_INCLUDE_USER_SETTING

public static final Setting<Boolean> ESQL_QUERYLOG_INCLUDE_USER_SETTING

GROK_WATCHDOG_MAX_EXECUTION_TIME

public static final Setting<TimeValue> GROK_WATCHDOG_MAX_EXECUTION_TIME

Maximum time (in milliseconds) that a GROK matcher is allowed to run before being interrupted. Limits how long a GROK matcher can run to protect against expensive regex patterns. Read directly from each node's own ClusterSettings wherever the matcher is built for execution (see LocalExecutionPlanner#planGrok) rather than being carried in the ES|QL Configuration wire format, since every node can resolve this node-scoped setting on its own — including live updates, since it is also dynamic.

STORED_FIELDS_SEQUENTIAL_PROPORTION

public static final Setting<Double> STORED_FIELDS_SEQUENTIAL_PROPORTION

Tuning parameter for deciding when to use the "merge" stored field loader. Think of it as "how similar to a sequential block of documents do I have to be before I'll use the merge reader?" So a value of 1 means I have to be exactly a sequential block, like 0, 1, 2, 3, .. 1299, 1300. A value of .2 means we'll use the sequential reader even if we only need one in ten documents.

The default value of this was experimentally derived using a script. And a little paranoia. A lower default value was looking good locally, but I'm concerned about the implications of effectively using this all the time.

Constructor Details

EsqlPlugin

public EsqlPlugin()

Method Details

externalBlobStorePool

public static String externalBlobStorePool()

Name of the dedicated thread pool backing all external blob-store access: metadata discovery (glob expansion, footer reads, and schema reconciliation performed by ExternalSourceResolver) as well as the blocking data reads and the streaming parse pipeline (segmentator + parser tasks of StreamingParallelParsingCoordinator) routed through OperatorFactoryRegistry#fileReadExecutor.

This is a separate pool from computePool() on purpose. These tasks block their thread on network I/O (a sequential decompressed stream read pulls compressed bytes from the object store) and on the parser's bounded hand-off queues; the compute Driver that consumes the parsed pages runs on computePool(). If the two shared a fixed pool, the segmentator plus parsing_parallelism parser tasks would occupy every slot and starve their own consumer, deadlocking the query (observed as a stalled heap-attack external query). Keeping them apart also prevents a single heavy external query from starving compute. In-flight cloud API calls are still bounded by the per-scheme permit semaphore in StorageProviderRegistry; the permits and this pool solve different problems — concurrency fairness/back-pressure versus thread isolation.

Must never resolve to ThreadPool.Names.SEARCH or ThreadPool.Names.GENERIC: a single heavy external query would otherwise starve regular searches or the rest of the node.

computePool

public static String computePool()

Name of the thread pool backing ES|QL compute — driver execution and the parallel worker fan-out. Returns the esql_worker pool. Distinct from externalBlobStorePool() so the blocking external I/O and parse pipeline cannot starve the compute drivers that consume its output (and vice versa).

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Overrides:

close in class Plugin

Throws:

IOException

createComponents

public Collection<?> createComponents(Plugin.PluginServices services)

Overrides:

createComponents in class Plugin

blockFactoryProvider

protected BlockFactoryProvider blockFactoryProvider(BlockFactoryBuilder builder)

getLicenseState

protected XPackLicenseState getLicenseState()

getSettings

public List<Setting<?>> getSettings()

The settings defined by the ESQL plugin.

Overrides:

getSettings in class Plugin

Returns:

the settings

getActions

public List<ActionPlugin.ActionHandler> getActions()

Specified by:

getActions in interface ActionPlugin

getRestHandlers

public List<RestHandler> getRestHandlers(ActionPlugin.RestHandlersServices restHandlersServices,
 Supplier<DiscoveryNodes> nodesInCluster,
 Predicate<NodeFeature> clusterSupportsFeature)

Specified by:

getRestHandlers in interface ActionPlugin

getNamedWriteables

public List<NamedWriteableRegistry.Entry> getNamedWriteables()

Overrides:

getNamedWriteables in class Plugin

getNamedXContent

public List<NamedXContentRegistry.Entry> getNamedXContent()

Overrides:

getNamedXContent in class Plugin

getExecutorBuilders

public List<ExecutorBuilder<?>> getExecutorBuilders(Settings settings)

Overrides:

getExecutorBuilders in class Plugin

loadExtensions

public void loadExtensions(ExtensiblePlugin.ExtensionLoader loader)

Specified by:

loadExtensions in interface ExtensiblePlugin

getGenericNamedWriteables

public List<SearchPlugin.GenericNamedWriteableSpec> getGenericNamedWriteables()

Specified by:

getGenericNamedWriteables in interface SearchPlugin