Package org.elasticsearch.xpack.esql.datasources.spi

package org.elasticsearch.xpack.esql.datasources.spi

Related Packages

Package	Description
org.elasticsearch.xpack.esql.datasources

Class	Description
AbstractMeteredStorageObject	Base for the leaf StorageObject providers (S3, GCS, Azure, HTTP, local): owns the AbstractMeteredStorageObject.counters field and the AbstractMeteredStorageObject.metrics() accessor that each used to re-declare identically, plus the shared native-async read completion handling (AbstractMeteredStorageObject.onReadComplete(java.util.concurrent.CompletableFuture<T>, java.util.function.BiConsumer<T, java.lang.Throwable>) / AbstractMeteredStorageObject.deliverRead(org.elasticsearch.action.ActionListener<org.elasticsearch.xpack.esql.datasources.spi.DirectReadBuffer>, org.elasticsearch.xpack.esql.datasources.spi.DirectReadBuffer, long)).
AggregatePushdownSupport	SPI for declaring that a format can compute aggregates from file-level statistics.
AggregatePushdownSupport.Pushability
AggregateSpec	Specification for a single aggregate to be computed from file-level statistics.
AggregateSpec.AggType
BufferingPageIterator	Base class for the external-format page iterators (NDJSON, CSV, parquet-rs) that buffer a single look-ahead Page in BufferingPageIterator.nextPage — materialized by the subclass's hasNext() and handed out (and nulled) by its next().
ColumnBlockConversions	Converts columnar primitive arrays to ESQL Blocks, avoiding the per-element overhead of Block.Builder when the source data is already in columnar form.
ColumnExtractor	Loads one projected column by physical row identity.
ColumnExtractorAware	Capability marker for FormatReader implementations that can re-read columns positionally after the primary forward scan.
ColumnExtractorProducer	Implemented by reader iterators that emit the synthetic ColumnExtractor.ROW_POSITION_COLUMN alongside their pages and can produce a matching ColumnExtractor whose addressing space is identical to the one the iterator is using for its row-position counter.
ConfigKeyValidator	Reject configuration keys not in any of the supplied claimed-key sets.
Configured<T>	Carrier for "configure with this map" operations: a value plus the keys it consumed.
Connector	A live connection to an external data source.
ConnectorFactory	Factory for creating connectors to external data sources.
DataSourceConfigDefinition	Metadata describing a datasource configuration field — its name, sensitivity, and value semantics.
DataSourceConfiguration	Base class for datasource configurations.
DataSourcePlugin	Extension point for data source implementations.
DataSourceValidationUtils	Helpers for DataSourceValidator implementations.
DataSourceValidator	Validates data source + dataset settings at CRUD time.
DeclaredTypeCoercions	The single source of truth for declared-type coercion on external datasets: which (physical file type → declared type) pairs an external reader coerces at read time, how a decoded physical block becomes a declared-type block (DeclaredTypeCoercions.castBlock(org.elasticsearch.compute.data.Block, org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.common.time.DateFormatter, org.elasticsearch.compute.data.BlockFactory, java.lang.String, org.elasticsearch.xpack.esql.datasources.spi.SkipWarnings)), and the one scalar conversion the string → date pair uses everywhere (DeclaredTypeCoercions.parseDatetimeMillis(java.lang.String, org.elasticsearch.common.time.DateFormatter)).
DecompressionCodec	SPI for decompression codecs.
DirectBufferFactory	Producer of DirectReadBuffers for StorageObject.readBytesAsync(long, long, org.elasticsearch.xpack.esql.datasources.spi.DirectBufferFactory, java.util.concurrent.Executor, org.elasticsearch.action.ActionListener<org.elasticsearch.xpack.esql.datasources.spi.DirectReadBuffer>).
DirectMemoryDebug	Use-after-free detection for off-heap (direct) read buffers.
DirectReadBuffer	Result of StorageObject.readBytesAsync(long, long, DirectBufferFactory, java.util.concurrent.Executor, org.elasticsearch.action.ActionListener): the bytes plus a Releasable that frees the native memory backing them.
DynamicThreshold	Per-query descriptor for the live TopN threshold exposed to format readers.
DynamicThresholdAware	Capability marker for FormatReader implementations that can use a live TopN threshold.
ErrorPolicy	Configures how format readers handle malformed or unparseable rows.
ErrorPolicy.Mode	Determines what happens to a row (or field) that fails to parse.
ExternalClientException	An external-source failure caused by the request rather than by the cluster: the data we were pointed at cannot be read or decoded (corrupt/truncated/malformed file, an unsupported format feature, a missing object).
ExternalException	Base type for failures raised while reading from an external data source — an object store, a file in some columnar/row format, a remote HTTP endpoint, etc.
ExternalServerException	An external-source failure caused by the cluster, not the request: a bug or a broken invariant in our own reading code (an unexpected internal state, a code path that should be unreachable).
ExternalSourceFactory	Common interface for complete external data source factories.
ExternalSourceMetrics	Node-level holder for the ES|QL external-data-source operational metrics, published through the node MeterRegistry (APM/OTLP) for serverless dashboards and alerts.
ExternalSplit	A serializable, parallelizable unit of work for an external data source.
ExternalUnavailableException	A retryable transport failure talking to the remote store backing an external data source — the store returned a 5xx, throttled us, or the connection timed out / was reset.
FileDataSourceConfiguration	Base configuration for file-based external sources (S3, GCS, Azure).
FileDataSourceConfiguration.AuthMode	The resolved credential-selection mode a provider builds a mechanism for.
FileDataSourceValidator	DataSourceValidator for file-based external sources (S3, GCS, Azure).
FileDataSourceValidator.FormatConfigKeyResolver	Resolves file-format metadata for dataset CRUD validation.
FileList	Indexed view over a resolved set of files from an external data source.
FilterPushdownSupport	SPI interface for filter pushdown support in external data sources.
FilterPushdownSupport.Pushability	Indicates whether an expression can be pushed to the data source.
FilterPushdownSupport.PushdownResult	Result of attempting to push filters to a data source.
FormatReadContext	Immutable context for a single FormatReader.read(org.elasticsearch.xpack.esql.datasources.spi.StorageObject, org.elasticsearch.xpack.esql.datasources.spi.FormatReadContext) or FormatReader.readAsync(org.elasticsearch.xpack.esql.datasources.spi.StorageObject, org.elasticsearch.xpack.esql.datasources.spi.FormatReadContext, java.util.concurrent.Executor, org.elasticsearch.action.ActionListener<org.elasticsearch.compute.operator.CloseableIterator<org.elasticsearch.compute.data.Page>>) call.
FormatReadContext.Builder
FormatReader	Unified interface for reading data formats.
FormatReader.SchemaResolution	Strategy for resolving schemas across multiple files in a glob/multi-file query.
FormatReaderFactory	Factory for creating FormatReader instances.
FormatReaderStatus	Typed per-format reader counter snapshot folded into the format_reader field of the external-source operator status.
FormatSpec	Describes a file format this plugin provides: a logical format name, the file extensions that select it, and the per-dataset configuration keys it recognises.
FrameIndex	Index of independently decompressible frames within a compressed file.
FrameIndex.FrameEntry
IndexedDecompressionCodec	Extension of DecompressionCodec for codecs that support index-based random access to independently compressed frames.
ListingHint	Metadata about a file already known from a directory listing, passed to ExternalSourceFactory.resolveMetadataAsync(java.lang.String, org.elasticsearch.xpack.esql.datasources.spi.ListingHint, java.util.Map<java.lang.String, java.lang.Object>, java.util.concurrent.Executor, org.elasticsearch.action.ActionListener<org.elasticsearch.xpack.esql.datasources.spi.SourceMetadata>) so the factory can build the storage object without a synchronous existence/HEAD probe before the async metadata read.
NoConfigFormatReader	Marker FormatReader for readers that recognise no per-query configuration keys.
NullSpliceRowPositionStrategy	Strategy for readers that have no native row-position channel and must surface _rowPosition as a NULL column (parquet-rs today; the Rust bridge does not yet expose row positions).
PassThroughRowPositionStrategy	Strategy for readers that natively populate the _rowPosition slot inside their own page-emitting iterator (parquet-mr, ORC, CSV, NDJSON).
QueryRequest	Describes a query to execute against a connector.
RangeAwareFormatReader	Extension of FormatReader for columnar formats (Parquet, ORC) that support row-group-level split parallelism.
RangeAwareFormatReader.SplitRange	A byte range within a file with optional per-range statistics (e.g.
RangeAwareFormatReader.SplitRef	Per-split descriptor passed to RangeAwareFormatReader.readAll(java.util.List<org.elasticsearch.xpack.esql.datasources.spi.RangeAwareFormatReader.SplitRef>, java.util.List<java.lang.String>, int).
RangeReadContext	Context for a single RangeAwareFormatReader.readRange(org.elasticsearch.xpack.esql.datasources.spi.StorageObject, org.elasticsearch.xpack.esql.datasources.spi.RangeReadContext) call.
RecordCappingInputStream	Stream wrapper that enforces the max_record_size byte cap on every record without decoding characters or visiting bytes outside the bulk-read fast path.
RecordSplitter	Finds format-specific record boundaries for splittable row-oriented external data sources.
ResultCursor	Streaming cursor over query results from a connector.
RowPositionStrategy	Per-reader strategy for populating the _rowPosition slot in emitted pages.
SegmentableFormatReader	Extension of FormatReader for line-oriented text formats (CSV, NDJSON) that support intra-file parallel parsing.
SimpleSourceMetadata	Simple immutable implementation of SourceMetadata.
SimpleSourceMetadata.Builder	Builder for SimpleSourceMetadata.
SkipWarnings	Collects response Warning headers for datasource read paths that skip a malformed input and resume processing (non-strict ErrorPolicy, or similar best-effort fallbacks).
SourceMetadata	Unified metadata output type returned by all schema discovery mechanisms.
SourceOperatorContext	Context for creating source operator factories.
SourceOperatorContext.Builder
SourceOperatorFactoryProvider	Functional interface for creating source operator factories.
SourceStatistics	Statistics about a data source for query planning and optimization.
SourceStatistics.ColumnStatistics	Statistics for an individual column.
Split	Represents a unit of work for a connector to execute.
SplitDiscoveryContext	Context passed to SplitProvider.discoverSplits(org.elasticsearch.xpack.esql.datasources.spi.SplitDiscoveryContext) containing all information needed to enumerate and optionally prune splits for an external source.
SplitDiscoveryResult	Result of SplitProvider.discoverSplits(org.elasticsearch.xpack.esql.datasources.spi.SplitDiscoveryContext): the discovered splits plus the post-prune "scanned" accounting reported in the query profile.
SplitProvider	Discovers parallelizable splits for an external data source.
SplitStats	Per-split statistics used by the optimizer for aggregate pushdown and filter pruning.
SplittableDecompressionCodec	Extension of DecompressionCodec for codecs that support splitting compressed files into independently decompressible ranges aligned to compressed block boundaries.
StorageObject	Unified interface for storage object access.
StorageObjectMetrics	Immutable snapshot of cumulative I/O counters for a StorageObject instance.
StorageObjectMetricsCounters	Mutable, thread-safe counter struct for storage I/O.
StoragePath	Represents a location in a storage system.
StorageProvider	Abstraction for accessing objects in external storage systems.
StorageProviderFactory	Factory for creating StorageProvider instances.
StorageProviderServices	Node-level services threaded into DataSourcePlugin.storageProviders(StorageProviderServices).
StripeColumnScope	How much per-stripe statistics a row-format external read should harvest while it scans, threaded to the readers through FormatReadContext.statsColumnScope().
TableCatalog	Connects to table catalog systems like Iceberg, Delta Lake, or Hudi.
TableCatalog.DataFile
TableCatalogFactory	Factory for creating TableCatalog instances.