Package org.elasticsearch.xpack.esql.plugin

Class TransportEsqlQueryAction

java.lang.Object

org.elasticsearch.action.support.TransportAction<EsqlQueryRequest,EsqlQueryResponse>

org.elasticsearch.action.support.HandledTransportAction<EsqlQueryRequest,EsqlQueryResponse>

org.elasticsearch.xpack.esql.plugin.TransportEsqlQueryAction

All Implemented Interfaces:

AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

public class TransportEsqlQueryAction
extends HandledTransportAction<EsqlQueryRequest,EsqlQueryResponse>
implements AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

Field Summary

Fields inherited from class org.elasticsearch.action.support.TransportAction

actionName, taskManager

Constructor Summary

Constructors

Constructor

Description

TransportEsqlQueryAction(TransportService transportService, ActionFilters actionFilters, PlanExecutor planExecutor, SearchService searchService, ExchangeService exchangeService, ClusterService clusterService, ViewResolver viewResolver, ProjectResolver projectResolver, ThreadPool threadPool, BigArrays bigArrays, BlockFactoryProvider blockFactoryProvider, Client client, NamedWriteableRegistry registry, IndexNameExpressionResolver indexNameExpressionResolver, UsageService usageService, UserAgentParserRegistry userAgentParserRegistry, IpLocationService ipLocationService, ActionLoggingFieldsProvider fieldProvider, ActivityLogWriterProvider logWriterProvider, CrossProjectModeDecider crossProjectModeDecider)

Method Summary

Modifier and Type	Method	Description
EsqlQueryTask	createTask(EsqlQueryRequest request, long id, String type, String action, TaskId parentTaskId, Map<String,String> headers, Map<String,String> originHeaders, AsyncExecutionId asyncExecutionId)
protected void	doExecute(Task task, EsqlQueryRequest request, ActionListener<EsqlQueryResponse> listener)
EnrichLookupService	enrichLookupService()
ExchangeService	exchangeService()
void	execute(EsqlQueryRequest request, EsqlQueryTask task, ActionListener<EsqlQueryResponse> listener)
protected Executor	externalBlobStoreExecutor()	Executor for external blob-store access: connector handshakes, registry wiring, source resolution (glob expansion, footer reads, schema reconciliation performed by ExternalSourceResolver), and the blocking data reads and streaming parse pipeline routed through OperatorFactoryRegistry.fileReadExecutor.
protected int	externalSourceConcurrency()	Maximum number of in-flight per-file metadata (footer) reads a single multi-file resolution may have outstanding, passed to ExternalSourceResolver as its fan-out bound.
LookupFromIndexService	getLookupFromIndexService()
EsqlQueryResponse	initialResponse(EsqlQueryTask task)
void	onFailureAfterTimeout(Exception exception)
void	onResponseAfterTimeout(EsqlQueryResponse response)
EsqlQueryResponse	readResponse(StreamInput inputStream)

Methods inherited from class org.elasticsearch.action.support.TransportAction

execute, executeDirect, localOnly

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TransportEsqlQueryAction

@Inject
public TransportEsqlQueryAction(TransportService transportService,
 ActionFilters actionFilters,
 PlanExecutor planExecutor,
 SearchService searchService,
 ExchangeService exchangeService,
 ClusterService clusterService,
 ViewResolver viewResolver,
 ProjectResolver projectResolver,
 ThreadPool threadPool,
 BigArrays bigArrays,
 BlockFactoryProvider blockFactoryProvider,
 Client client,
 NamedWriteableRegistry registry,
 IndexNameExpressionResolver indexNameExpressionResolver,
 UsageService usageService,
 UserAgentParserRegistry userAgentParserRegistry,
 IpLocationService ipLocationService,
 ActionLoggingFieldsProvider fieldProvider,
 ActivityLogWriterProvider logWriterProvider,
 CrossProjectModeDecider crossProjectModeDecider)

Method Details

externalBlobStoreExecutor

protected Executor externalBlobStoreExecutor()

Executor for external blob-store access: connector handshakes, registry wiring, source resolution (glob expansion, footer reads, schema reconciliation performed by ExternalSourceResolver), and the blocking data reads and streaming parse pipeline routed through OperatorFactoryRegistry.fileReadExecutor.

Backed by EsqlPlugin.externalBlobStorePool() — the dedicated esql_external_io scaling pool, sized 0..ExternalSourceSettings.blobStoreConcurrency(org.elasticsearch.common.settings.Settings) (the single CPU-scaled concurrency knob). It is deliberately separate from the esql_worker compute pool (EsqlPlugin.computePool()): the blocking parse pipeline (segmentator + parser tasks) must not occupy the same threads as the compute drivers that consume its output, or the parser starves its consumer and deadlocks the query. Isolated from ThreadPool.Names.SEARCH to prevent heavy external queries (glob expansion over thousands of files, S3 footer reads) from starving regular ES searches — the reported production regression. The resolver's join pattern needs up to MAX_PARALLEL_METADATA_READS + 1 running slots; on nodes where that exceeds the pool size the BoundedParallelGather runner throttles submission rather than overflowing the queue, and on saturation across concurrent ES|QL queries it fails fast per-slot rather than deadlocking. In-flight cloud API calls are additionally bounded by the per-scheme permit semaphore in StorageProviderRegistry — permits govern concurrency fairness, the pool governs thread isolation.

This method is the coordinator-side hook: overriding it lets tests or a future re-routing move external blob-store access to a different pool without touching call sites.

externalSourceConcurrency

protected int externalSourceConcurrency()

Maximum number of in-flight per-file metadata (footer) reads a single multi-file resolution may have outstanding, passed to ExternalSourceResolver as its fan-out bound. Because footer reads are async (the esql_worker thread is released across the read), this caps concurrent in-flight reads rather than pinning that many threads, so the bound may safely exceed the pool size. It is the shared ExternalSourceSettings.blobStoreConcurrency(org.elasticsearch.common.settings.Settings) value — the single effective blob-store access concurrency that the data-read path also reads — so discovery throttles its footer fan-out with the same node-size-scaled formula (snapshot_meta shape, capped at 100, and any operator override once that setting lands) instead of the raw esql_worker.getMax() pool size.

doExecute

protected void doExecute(Task task,
 EsqlQueryRequest request,
 ActionListener<EsqlQueryResponse> listener)

Specified by:

doExecute in class TransportAction<EsqlQueryRequest,EsqlQueryResponse>

execute

public void execute(EsqlQueryRequest request,
 EsqlQueryTask task,
 ActionListener<EsqlQueryResponse> listener)

Specified by:

execute in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

onResponseAfterTimeout

public void onResponseAfterTimeout(EsqlQueryResponse response)

Specified by:

onResponseAfterTimeout in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

onFailureAfterTimeout

public void onFailureAfterTimeout(Exception exception)

Specified by:

onFailureAfterTimeout in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

exchangeService

public ExchangeService exchangeService()

enrichLookupService

public EnrichLookupService enrichLookupService()

createTask

public EsqlQueryTask createTask(EsqlQueryRequest request,
 long id,
 String type,
 String action,
 TaskId parentTaskId,
 Map<String,String> headers,
 Map<String,String> originHeaders,
 AsyncExecutionId asyncExecutionId)

Specified by:

createTask in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

initialResponse

public EsqlQueryResponse initialResponse(EsqlQueryTask task)

Specified by:

initialResponse in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

readResponse

public EsqlQueryResponse readResponse(StreamInput inputStream)
                               throws IOException

Specified by:

readResponse in interface AsyncTaskManagementService.AsyncOperation<EsqlQueryRequest,EsqlQueryResponse,EsqlQueryTask>

Throws:

IOException

getLookupFromIndexService

public LookupFromIndexService getLookupFromIndexService()