Package org.elasticsearch.xpack.esql.datasources

Class SyntheticColumns

java.lang.Object

org.elasticsearch.xpack.esql.datasources.SyntheticColumns

public final class SyntheticColumns
extends Object

Helpers for reader-synthesized internal channels — columns with no backing source field, filled by the format reader from a per-format value source. Today the only synthetic kind is SyntheticColumns.Kind.ROW_POSITION (column name ColumnExtractor.ROW_POSITION_COLUMN), which the optimizer (InjectRowPositionForExternalId) injects whenever _id or _file.record_ref is requested; format readers fill the slot from their own per-format source (CSV counter, ORC RecordReader#getRowNumber, NDJSON byte offset, parquet-mr packed extractor id) and VirtualColumnIterator consumes it before the data reaches the user.

Distinct from virtual columns: virtual columns are user-visible names you can write in a METADATA clause (the MetadataAttribute.ATTRIBUTES_MAP names — _index, _id, _version, … — plus the _file.* family); they implement the VirtualAttribute marker. Synthetic columns are internal-only: the user cannot name _rowPosition in a query, but its value reaches the user via the virtual _file.record_ref and through _id's composition.

The SyntheticColumns.Kind registry is the single source of truth for the set of synthetic columns, their canonical names, and their attribute shapes — consumers derive a kind's behavior from its accessors rather than dispatching per member.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	SyntheticColumns.Kind	Registry of reader-synthesized internal channels.

Field Summary

Fields

Modifier and Type	Field	Description
static final Set<String>	NAMES	Names of every reader-synthesized internal channel.

Method Summary

Modifier and Type	Method	Description
static SyntheticColumns.Kind	kindOf(String name)	The SyntheticColumns.Kind matching name, or null when name is not synthetic.
static Attribute	newAttribute(SyntheticColumns.Kind kind)	Reader-side attribute shape for the slot of kind: a ReferenceAttribute whose type and nullability come from the SyntheticColumns.Kind record.
static Attribute	newRowPositionAttribute()	Shorthand for newAttribute(Kind.ROW_POSITION); kept for the kind-specific call sites.
static MetadataAttribute	newRowPositionMetadataAttribute(Source source)	Optimizer-side attribute shape for the SyntheticColumns.Kind.ROW_POSITION slot: a MetadataAttribute typed and nullability-marked from the SyntheticColumns.Kind record, marked synthetic.
static int	rowPositionIndexInAttributes(List<? extends Attribute> attributes)	Index of ColumnExtractor.ROW_POSITION_COLUMN in attributes, by attribute name; -1 when absent or the list is null.
static int	rowPositionIndexInNames(List<String> projectedColumns)	Index of ColumnExtractor.ROW_POSITION_COLUMN in projectedColumns, or -1 when absent or the list is null.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

NAMES

public static final Set<String> NAMES

Names of every reader-synthesized internal channel. The producer pipeline injects these, and rendering code (e.g. SynthesizeExternalSource) excludes them from the rendered _source object. User data columns whose names happen to start with _ (e.g. Spark's _corrupt_record, a user-supplied _status) are real data — a leading underscore on its own is not the filter; membership in this set is.

Method Details

kindOf

public static SyntheticColumns.Kind kindOf(String name)

The SyntheticColumns.Kind matching name, or null when name is not synthetic.

rowPositionIndexInNames

public static int rowPositionIndexInNames(List<String> projectedColumns)

Index of ColumnExtractor.ROW_POSITION_COLUMN in projectedColumns, or -1 when absent or the list is null.

rowPositionIndexInAttributes

public static int rowPositionIndexInAttributes(List<? extends Attribute> attributes)

Index of ColumnExtractor.ROW_POSITION_COLUMN in attributes, by attribute name; -1 when absent or the list is null.

newAttribute

public static Attribute newAttribute(SyntheticColumns.Kind kind)

Reader-side attribute shape for the slot of kind: a ReferenceAttribute whose type and nullability come from the SyntheticColumns.Kind record. Used by format readers when they synthesize the projected attribute for a slot that has no source column behind it (e.g. the _rowPosition channel in CsvFormatReader's switch on SyntheticColumns.Kind).

newRowPositionAttribute

public static Attribute newRowPositionAttribute()

Shorthand for newAttribute(Kind.ROW_POSITION); kept for the kind-specific call sites.

newRowPositionMetadataAttribute

public static MetadataAttribute newRowPositionMetadataAttribute(Source source)

Optimizer-side attribute shape for the SyntheticColumns.Kind.ROW_POSITION slot: a MetadataAttribute typed and nullability-marked from the SyntheticColumns.Kind record, marked synthetic. Paired with newRowPositionAttribute() so type and nullability stay in lock-step between the optimizer injection site and the reader synthesis site.