Package org.elasticsearch.xpack.esql.datasources.spi

Record Class SourceOperatorContext

java.lang.Object
java.lang.Record
org.elasticsearch.xpack.esql.datasources.spi.SourceOperatorContext
public record SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, int rowLimit, Executor executor, Executor fileReadExecutor, Map<String,Object> config, Map<String,Object> sourceMetadata, Object pushedFilter, List<Expression> pushedExpressions, FileList fileList, Map<StoragePath,SchemaReconciliation.FileSchemaInfo> schemaMap, ExternalSplit split, Set<String> partitionColumnNames, ExternalSliceQueue sliceQueue, int parsingParallelism, int maxConcurrentOpenSegments, int maxRecordBytes, int parallelism, String datasetName, boolean deferredExtraction, DeclaredReadSpec declaredReadSpec) extends Record
Context for creating source operator factories. Uses Java record for immutability and automatic equals/hashCode/toString.

Note: Record accessors have no "get" prefix per project conventions.

This context is passed to SourceOperatorFactoryProvider implementations to provide all necessary information for creating a source operator factory.

For table-based sources (Iceberg, Delta Lake), the sourceMetadata() map contains opaque source-specific data (like native schema) that the operator factory needs but core doesn't interpret.

The pushedFilter() contains an opaque filter object that was pushed down during optimization. Since external sources execute on coordinator only, this filter is never serialized - it's created during local physical optimization and consumed immediately by the operator factory in the same JVM.

  • Field Details

    • DEFAULT_MAX_CONCURRENT_OPEN_SEGMENTS

      public static final int DEFAULT_MAX_CONCURRENT_OPEN_SEGMENTS
      Single source of truth for the max_concurrent_open_segments default. Lives in this SPI (leaf) layer so both the QueryPragmas setting and the datasources-side fallback defaults reference it without datasources having to depend on plugin. Change here and it propagates.
      See Also:

  • Constructor Details

    • SourceOperatorContext

      public SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, int rowLimit, Executor executor, @Nullable Executor fileReadExecutor, Map<String,Object> config, Map<String,Object> sourceMetadata, Object pushedFilter, List<Expression> pushedExpressions, FileList fileList, Map<StoragePath,SchemaReconciliation.FileSchemaInfo> schemaMap, @Nullable ExternalSplit split, Set<String> partitionColumnNames, @Nullable ExternalSliceQueue sliceQueue, int parsingParallelism, int maxConcurrentOpenSegments, int maxRecordBytes, int parallelism, @Nullable String datasetName, boolean deferredExtraction, DeclaredReadSpec declaredReadSpec)
      Creates an instance of a SourceOperatorContext record class.
      Parameters:
      sourceType - the value for the sourceType record component
      path - the value for the path record component
      projectedColumns - the value for the projectedColumns record component
      attributes - the value for the attributes record component
      batchSize - the value for the batchSize record component
      maxBufferSize - the value for the maxBufferSize record component
      rowLimit - the value for the rowLimit record component
      executor - the value for the executor record component
      fileReadExecutor - the value for the fileReadExecutor record component
      config - the value for the config record component
      sourceMetadata - the value for the sourceMetadata record component
      pushedFilter - the value for the pushedFilter record component
      pushedExpressions - the value for the pushedExpressions record component
      fileList - the value for the fileList record component
      schemaMap - the value for the schemaMap record component
      split - the value for the split record component
      partitionColumnNames - the value for the partitionColumnNames record component
      sliceQueue - the value for the sliceQueue record component
      parsingParallelism - the value for the parsingParallelism record component
      maxConcurrentOpenSegments - the value for the maxConcurrentOpenSegments record component
      maxRecordBytes - the value for the maxRecordBytes record component
      parallelism - the value for the parallelism record component
      datasetName - the value for the datasetName record component
      deferredExtraction - the value for the deferredExtraction record component
      declaredReadSpec - the value for the declaredReadSpec record component

    • SourceOperatorContext

      public SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, Executor executor, Map<String,Object> config, Map<String,Object> sourceMetadata, Object pushedFilter, FileList fileList, @Nullable ExternalSplit split)

    • SourceOperatorContext

      public SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, Executor executor, Map<String,Object> config, Map<String,Object> sourceMetadata, Object pushedFilter, FileList fileList)

    • SourceOperatorContext

      public SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, Executor executor, Map<String,Object> config, Map<String,Object> sourceMetadata, Object pushedFilter)

    • SourceOperatorContext

      public SourceOperatorContext(String sourceType, StoragePath path, List<String> projectedColumns, List<Attribute> attributes, int batchSize, int maxBufferSize, Executor executor, Map<String,Object> config)

  • Method Details

    • builder

      public static SourceOperatorContext.Builder builder()

    • toString

      public final String toString()
      Returns a string representation of this record class. The representation contains the name of the class, followed by the name and value of each of the record components.
      Specified by:
      toString in class Record
      Returns:
      a string representation of this object

    • hashCode

      public final int hashCode()
      Returns a hash code value for this object. The value is derived from the hash code of each of the record components.
      Specified by:
      hashCode in class Record
      Returns:
      a hash code value for this object

    • equals

      public final boolean equals(Object o)
      Indicates whether some other object is "equal to" this one. The objects are equal if the other object is of the same class and if all the record components are equal. Reference components are compared with Objects::equals(Object,Object); primitive components are compared with '=='.
      Specified by:
      equals in class Record
      Parameters:
      o - the object with which to compare
      Returns:
      true if this object is the same as the o argument; false otherwise.

    • sourceType

      public String sourceType()
      Returns the value of the sourceType record component.
      Returns:
      the value of the sourceType record component

    • path

      public StoragePath path()
      Returns the value of the path record component.
      Returns:
      the value of the path record component

    • projectedColumns

      public List<String> projectedColumns()
      Returns the value of the projectedColumns record component.
      Returns:
      the value of the projectedColumns record component

    • attributes

      public List<Attribute> attributes()
      Returns the value of the attributes record component.
      Returns:
      the value of the attributes record component

    • batchSize

      public int batchSize()
      Returns the value of the batchSize record component.
      Returns:
      the value of the batchSize record component

    • maxBufferSize

      public int maxBufferSize()
      Returns the value of the maxBufferSize record component.
      Returns:
      the value of the maxBufferSize record component

    • rowLimit

      public int rowLimit()
      Returns the value of the rowLimit record component.
      Returns:
      the value of the rowLimit record component

    • executor

      public Executor executor()
      Returns the value of the executor record component.
      Returns:
      the value of the executor record component

    • fileReadExecutor

      @Nullable public Executor fileReadExecutor()
      Returns the value of the fileReadExecutor record component.
      Returns:
      the value of the fileReadExecutor record component

    • config

      public Map<String,Object> config()
      Returns the value of the config record component.
      Returns:
      the value of the config record component

    • sourceMetadata

      public Map<String,Object> sourceMetadata()
      Returns the value of the sourceMetadata record component.
      Returns:
      the value of the sourceMetadata record component

    • pushedFilter

      public Object pushedFilter()
      Returns the value of the pushedFilter record component.
      Returns:
      the value of the pushedFilter record component

    • pushedExpressions

      public List<Expression> pushedExpressions()
      Returns the value of the pushedExpressions record component.
      Returns:
      the value of the pushedExpressions record component

    • fileList

      public FileList fileList()
      Returns the value of the fileList record component.
      Returns:
      the value of the fileList record component

    • schemaMap

      public Map<StoragePath,SchemaReconciliation.FileSchemaInfo> schemaMap()
      Returns the value of the schemaMap record component.
      Returns:
      the value of the schemaMap record component

    • split

      @Nullable public ExternalSplit split()
      Returns the value of the split record component.
      Returns:
      the value of the split record component

    • partitionColumnNames

      public Set<String> partitionColumnNames()
      Returns the value of the partitionColumnNames record component.
      Returns:
      the value of the partitionColumnNames record component

    • sliceQueue

      @Nullable public ExternalSliceQueue sliceQueue()
      Returns the value of the sliceQueue record component.
      Returns:
      the value of the sliceQueue record component

    • parsingParallelism

      public int parsingParallelism()
      Returns the value of the parsingParallelism record component.
      Returns:
      the value of the parsingParallelism record component

    • maxConcurrentOpenSegments

      public int maxConcurrentOpenSegments()
      Returns the value of the maxConcurrentOpenSegments record component.
      Returns:
      the value of the maxConcurrentOpenSegments record component

    • maxRecordBytes

      public int maxRecordBytes()
      Returns the value of the maxRecordBytes record component.
      Returns:
      the value of the maxRecordBytes record component

    • parallelism

      public int parallelism()
      Returns the value of the parallelism record component.
      Returns:
      the value of the parallelism record component

    • datasetName

      @Nullable public String datasetName()
      Returns the value of the datasetName record component.
      Returns:
      the value of the datasetName record component

    • deferredExtraction

      public boolean deferredExtraction()
      Returns the value of the deferredExtraction record component.
      Returns:
      the value of the deferredExtraction record component

    • declaredReadSpec

      public DeclaredReadSpec declaredReadSpec()
      Returns the value of the declaredReadSpec record component.
      Returns:
      the value of the declaredReadSpec record component