Class BulkInferenceExecutor

java.lang.Object
org.elasticsearch.xpack.esql.inference.bulk.BulkInferenceExecutor

public class BulkInferenceExecutor extends Object
Executes a sequence of inference requests in bulk with throttling and concurrency control.
  • Constructor Details

    • BulkInferenceExecutor

      public BulkInferenceExecutor(InferenceRunner inferenceRunner, ThreadPool threadPool, BulkInferenceExecutionConfig bulkExecutionConfig)
      Constructs a new BulkInferenceExecutor.
      Parameters:
      inferenceRunner - The inference runner used to execute individual inference requests.
      threadPool - The thread pool for executing inference tasks.
      bulkExecutionConfig - Configuration options (throttling and concurrency limits).
  • Method Details

    • execute

      public void execute(BulkInferenceRequestIterator requests, ActionListener<List<InferenceAction.Response>> listener)
      Executes multiple inference requests in bulk and collects all responses.
      Parameters:
      requests - An iterator over the inference requests to execute
      listener - Called with the list of all responses in request order
    • execute

      public void execute(BulkInferenceRequestIterator requests, Consumer<InferenceAction.Response> responseConsumer, ActionListener<Void> completionListener)
      Executes multiple inference requests in bulk with streaming response handling.

      This method orchestrates the entire bulk inference process: 1. Creates execution state to track progress and responses 2. Sets up response handling pipeline 3. Initiates asynchronous request processing

      Parameters:
      requests - An iterator over the inference requests to execute
      responseConsumer - Called for each successful inference response as they complete
      completionListener - Called when all requests are complete or if any error occurs