Class InferenceOperator

All Implemented Interfaces:
Closeable, AutoCloseable, Operator, org.elasticsearch.core.Releasable
Direct Known Subclasses:
CompletionOperator, RerankOperator

public abstract class InferenceOperator extends AsyncOperator<InferenceOperator.OngoingInferenceResult>
An abstract asynchronous operator that performs throttled bulk inference execution using an InferenceRunner.

The InferenceOperator integrates with the compute framework supports throttled bulk execution of inference requests. It transforms input Page into inference requests, asynchronously executes them, and converts the responses into a new Page.