Module org.elasticsearch.xcore

Package org.elasticsearch.xpack.core.inference.chunking

Class EmbeddingRequestChunker<E extends EmbeddingResults.Embedding<E>>

java.lang.Object

org.elasticsearch.xpack.core.inference.chunking.EmbeddingRequestChunker<E>

public class EmbeddingRequestChunker<E extends EmbeddingResults.Embedding<E>>
extends Object

This class creates optimally sized batches of input strings for batched processing splitting long strings into smaller chunks. Multiple inputs may be fit into a single batch or a single large input that has been chunked may spread over multiple batches.

The final aspect is to gather the responses from the batch processing and map the results back to the original element in the input list.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static final record	EmbeddingRequestChunker.BatchRequest
static final record	EmbeddingRequestChunker.BatchRequestAndListener

Constructor Summary

Constructors

Constructor	Description
EmbeddingRequestChunker(List<ChunkInferenceInput> inputs, int maxNumberOfInputsPerBatch)
EmbeddingRequestChunker(List<ChunkInferenceInput> inputs, int maxNumberOfInputsPerBatch, boolean batchChunksAcrossInputs, ChunkingSettings defaultChunkingSettings)
EmbeddingRequestChunker(List<ChunkInferenceInput> inputs, int maxNumberOfInputsPerBatch, int wordsPerChunk, int chunkOverlap)
EmbeddingRequestChunker(List<ChunkInferenceInput> inputs, int maxNumberOfInputsPerBatch, ChunkingSettings defaultChunkingSettings)

Method Summary

Modifier and Type	Method	Description
List<EmbeddingRequestChunker.BatchRequestAndListener>	batchRequestsWithListeners(ActionListener<List<ChunkedInference>> finalListener)	Returns a list of batched inputs and a ActionListener for each batch.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

EmbeddingRequestChunker

public EmbeddingRequestChunker(List<ChunkInferenceInput> inputs,
 int maxNumberOfInputsPerBatch)

EmbeddingRequestChunker

public EmbeddingRequestChunker(List<ChunkInferenceInput> inputs,
 int maxNumberOfInputsPerBatch,
 int wordsPerChunk,
 int chunkOverlap)

EmbeddingRequestChunker

public EmbeddingRequestChunker(List<ChunkInferenceInput> inputs,
 int maxNumberOfInputsPerBatch,
 ChunkingSettings defaultChunkingSettings)

EmbeddingRequestChunker

public EmbeddingRequestChunker(List<ChunkInferenceInput> inputs,
 int maxNumberOfInputsPerBatch,
 boolean batchChunksAcrossInputs,
 ChunkingSettings defaultChunkingSettings)

Method Details

batchRequestsWithListeners

public List<EmbeddingRequestChunker.BatchRequestAndListener> batchRequestsWithListeners(ActionListener<List<ChunkedInference>> finalListener)

Returns a list of batched inputs and a ActionListener for each batch.

Parameters:

finalListener - The listener to call once all the batches are processed

Returns:

Batches and listeners