Class AbstractLookupService<R extends org.elasticsearch.xpack.esql.enrich.AbstractLookupService.Request,T extends org.elasticsearch.xpack.esql.enrich.AbstractLookupService.TransportRequest>
- Direct Known Subclasses:
EnrichLookupService,LookupFromIndexService
AbstractLookupService performs a LEFT JOIN for a given input
page against another index that must have only a single
shard.
This registers a TransportRequestHandler so we can handle requests
to join data that isn't local to the node, but it is much faster if the
data is already local.
The join process spawns a Driver per incoming page which runs in
two or three stages:
Stage 1: Finding matching document IDs for the input page. This stage is done
by the EnrichQuerySourceOperator. The output page of this stage is
represented as [DocVector, IntBlock: positions of the input terms].
Stage 2: Extracting field values for the matched document IDs. The output page
is represented as
[DocVector, IntBlock: positions, Block: field1, Block: field2,...].
Stage 3: Optionally this combines the extracted values based on positions and filling
nulls for positions without matches. This is done by MergePositionsOperator.
The output page is represented as [Block: field1, Block: field2,...].
The Page.getPositionCount() of the output Page is equal to the
Page.getPositionCount() of the input page. In other words - it returns
the same number of rows that it was sent no matter how many documents match.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic final recordAbstractLookupServiceuses this to power the queries and field loading that it needs to perform to actually do the lookup.static interfaceCreate aAbstractLookupService.LookupShardContextfor a locally allocatedShardId. -
Field Summary
FieldsModifier and TypeFieldDescriptionprotected final ClusterServiceprotected final Executorprotected final TransportService -
Method Summary
Modifier and TypeMethodDescriptionprotected abstract org.elasticsearch.xpack.esql.enrich.AbstractLookupService.LookupResponsecreateLookupResponse(List<Page> resultPages, BlockFactory blockFactory) Build the response.final voidlookupAsync(R request, CancellableTask parentTask, ActionListener<List<Page>> outListener) Perform the actual lookup.protected abstract QueryListqueryList(T request, SearchExecutionContext context, Block inputBlock, DataType inputDataType) Build a list of queries to perform inside the actual lookup.protected abstract org.elasticsearch.xpack.esql.enrich.AbstractLookupService.LookupResponsereadLookupResponse(StreamInput in, BlockFactory blockFactory) Read the response from aStreamInput.protected voidsendChildRequest(CancellableTask parentTask, ActionListener<List<Page>> delegate, DiscoveryNode targetNode, T transportRequest) protected static QueryListtermQueryList(MappedFieldType field, SearchExecutionContext searchExecutionContext, Block block, DataType inputDataType) protected abstract TtransportRequest(R request, ShardId shardId) Convert a request as sent tolookupAsync(R, org.elasticsearch.tasks.CancellableTask, org.elasticsearch.action.ActionListener<java.util.List<org.elasticsearch.compute.data.Page>>)into a transport request after preflight checks have been performed.
-
Field Details
-
clusterService
-
transportService
-
executor
-
-
Method Details
-
getThreadContext
-
transportRequest
Convert a request as sent tolookupAsync(R, org.elasticsearch.tasks.CancellableTask, org.elasticsearch.action.ActionListener<java.util.List<org.elasticsearch.compute.data.Page>>)into a transport request after preflight checks have been performed. -
queryList
protected abstract QueryList queryList(T request, SearchExecutionContext context, Block inputBlock, DataType inputDataType) Build a list of queries to perform inside the actual lookup. -
createLookupResponse
protected abstract org.elasticsearch.xpack.esql.enrich.AbstractLookupService.LookupResponse createLookupResponse(List<Page> resultPages, BlockFactory blockFactory) throws IOException Build the response.- Throws:
IOException
-
readLookupResponse
protected abstract org.elasticsearch.xpack.esql.enrich.AbstractLookupService.LookupResponse readLookupResponse(StreamInput in, BlockFactory blockFactory) throws IOException Read the response from aStreamInput.- Throws:
IOException
-
termQueryList
protected static QueryList termQueryList(MappedFieldType field, SearchExecutionContext searchExecutionContext, Block block, DataType inputDataType) -
lookupAsync
public final void lookupAsync(R request, CancellableTask parentTask, ActionListener<List<Page>> outListener) Perform the actual lookup. -
sendChildRequest
protected void sendChildRequest(CancellableTask parentTask, ActionListener<List<Page>> delegate, DiscoveryNode targetNode, T transportRequest)
-