Class BulkInferenceExecutor
java.lang.Object
org.elasticsearch.xpack.esql.inference.bulk.BulkInferenceExecutor
Executes a sequence of inference requests in bulk with throttling and concurrency control.
-
Constructor Summary
ConstructorsConstructorDescriptionBulkInferenceExecutor(InferenceRunner inferenceRunner, ThreadPool threadPool, BulkInferenceExecutionConfig bulkExecutionConfig) Constructs a newBulkInferenceExecutor. -
Method Summary
Modifier and TypeMethodDescriptionvoidexecute(BulkInferenceRequestIterator requests, Consumer<InferenceAction.Response> responseConsumer, ActionListener<Void> completionListener) Executes multiple inference requests in bulk with streaming response handling.voidexecute(BulkInferenceRequestIterator requests, ActionListener<List<InferenceAction.Response>> listener) Executes multiple inference requests in bulk and collects all responses.
-
Constructor Details
-
BulkInferenceExecutor
public BulkInferenceExecutor(InferenceRunner inferenceRunner, ThreadPool threadPool, BulkInferenceExecutionConfig bulkExecutionConfig) Constructs a newBulkInferenceExecutor.- Parameters:
inferenceRunner- The inference runner used to execute individual inference requests.threadPool- The thread pool for executing inference tasks.bulkExecutionConfig- Configuration options (throttling and concurrency limits).
-
-
Method Details
-
execute
public void execute(BulkInferenceRequestIterator requests, ActionListener<List<InferenceAction.Response>> listener) Executes multiple inference requests in bulk and collects all responses.- Parameters:
requests- An iterator over the inference requests to executelistener- Called with the list of all responses in request order
-
execute
public void execute(BulkInferenceRequestIterator requests, Consumer<InferenceAction.Response> responseConsumer, ActionListener<Void> completionListener) Executes multiple inference requests in bulk with streaming response handling.This method orchestrates the entire bulk inference process: 1. Creates execution state to track progress and responses 2. Sets up response handling pipeline 3. Initiates asynchronous request processing
- Parameters:
requests- An iterator over the inference requests to executeresponseConsumer- Called for each successful inference response as they completecompletionListener- Called when all requests are complete or if any error occurs
-