Package org.elasticsearch.xpack.esql.datasources.spi

Interface ColumnExtractorProducer

All Known Implementing Classes:
StatsCapturingIterator
public interface ColumnExtractorProducer
Implemented by reader iterators that emit the synthetic ColumnExtractor.ROW_POSITION_COLUMN alongside their pages and can produce a matching ColumnExtractor whose addressing space is identical to the one the iterator is using for its row-position counter.

Implementing this interface is mandatory for any reader iterator on the deferred-extraction path: opting in commits to the full handshake — the iterator both produces a matching extractor and emits _rowPosition values already encoded with the registry- assigned id, so downstream operators can decode them without an intermediate re-encoding pass. Iterators that don't support deferred extraction simply omit ColumnExtractorAware on the format reader and never reach this code path.

Encoding handshake

  1. The factory wraps the iterator's pages and calls createColumnExtractor() to build a matching ColumnExtractor.
  2. The factory registers the extractor with SourceExtractors, receiving an id.
  3. The factory calls setExtractorId(int) before draining the first page, handing the iterator the id it must OR into every _rowPosition value it emits.
  4. From then on, every page the iterator returns carries pre-encoded _rowPosition values of the form (id << 48) | physicalRowOffset — see SourceExtractors for the bit layout.
Encoding inside the iterator is significantly cheaper than wrapping the page stream with a separate encoder: it avoids re-allocating the _rowPosition block per page (the iterator already has the values in a primitive long[] buffer) and removes a per-page page-rebuild step from the producer thread.

Implementations should construct the extractor lazily — typically on the first call to createColumnExtractor() — and may return the same instance on subsequent calls. Lifetime is owned by the caller via SourceExtractors (the registry calls Releasable.close() when the driver finishes).

  • Method Details

    • createColumnExtractor

      ColumnExtractor createColumnExtractor() throws IOException
      Creates the ColumnExtractor matching this iterator's addressing space.

      The iterator must already be positioned to emit pages with ColumnExtractor.ROW_POSITION_COLUMN before this is called; implementations may capture iterator-internal state (such as a range-restricted footer) at construction time.

      Throws:
      IOException

    • setExtractorId

      void setExtractorId(int id)
      Hands the iterator the SourceExtractors-assigned id under which its matching ColumnExtractor is registered. Must be called once, after createColumnExtractor() and before the first page is drained. Every subsequent page the iterator emits must carry _rowPosition values already encoded with this id (typically by OR-ing ((long) id << 48) into each value as it is materialised).

      Calling this method twice on the same iterator, or skipping it on the deferred path, is a programmer error: the iterator's pages either carry mismatched ids or unencoded raw row offsets that the lookup registry cannot route.

      Parameters:
      id - registry-assigned extractor id; must be in [0, MAX_EXTRACTOR_ID] (see SourceExtractors)