Package org.elasticsearch.xpack.esql.datasources

Class ExternalMetadataColumns

java.lang.Object

org.elasticsearch.xpack.esql.datasources.ExternalMetadataColumns

public final class ExternalMetadataColumns
extends Object

Registry of the standard ES index metadata names (_id, _index, _version, ...) that the external-source pipeline knows how to materialise on external datasets. The data types are sourced from MetadataAttribute.ATTRIBUTES_MAP so the binding here and in the analyzer always agree (including snapshot-only entries such as _tier).

Sibling to FileMetadataColumns (_file.*). Both families are request-driven (the user names them in METADATA) and materialized by VirtualColumnIterator on the producer thread; the split exists because _file.* comes from per-file stat while the standard names route through MetadataAttribute.ATTRIBUTES_MAP.

Two materialisation lanes exist: see PER_FILE_CONSTANT_NAMES for the canonical per-file-constant set; the remaining standard names (_id via ExternalRowIdentity, _source via SynthesizeExternalSource) are per-row composed.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	ID
static final String	IGNORED
static final String	INDEX
static final String	INDEX_MODE
static final Set<String>	PER_FILE_CONSTANT_NAMES	Names of standard metadata columns that are materialised by the producer-side constant-block path (per-file values, including SQL NULL where unavailable).
static final Set<String>	RESERVED_NAMES	The dedicated metadata namespace for reservation/rename purposes: STANDARD_NAMES plus every standard name that is only snapshot-gated for binding (_tier).
static final String	SCORE
static final String	SIZE
static final String	SLICE
static final String	SOURCE
static final Set<String>	STANDARD_NAMES	Every standard metadata name an external relation can bind — PER_FILE_CONSTANT_NAMES plus the per-row composed pair (ID, SOURCE).
static final String	TSID
static final String	VERSION

Method Summary

Modifier and Type	Method	Description
static Map<String,Object>	extractPerFileConstants(String datasetName, Long lastModifiedMillis)	Variant for callers that already hold the file's last-modified epoch-millis (e.g.
static Map<String,Object>	extractPerFileConstants(String datasetName, FileList fileList, int index)	Build the per-file constant values for the standard metadata names listed in PER_FILE_CONSTANT_NAMES.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

ID

public static final String ID

See Also:

• Constant Field Values

INDEX

public static final String INDEX

See Also:

• Constant Field Values

VERSION

public static final String VERSION

See Also:

• Constant Field Values

SCORE

public static final String SCORE

See Also:

• Constant Field Values

SOURCE

public static final String SOURCE

See Also:

• Constant Field Values

IGNORED

public static final String IGNORED

See Also:

• Constant Field Values

INDEX_MODE

public static final String INDEX_MODE

See Also:

• Constant Field Values

TSID

public static final String TSID

See Also:

• Constant Field Values

SIZE

public static final String SIZE

See Also:

• Constant Field Values

SLICE

public static final String SLICE

See Also:

• Constant Field Values

PER_FILE_CONSTANT_NAMES

public static final Set<String> PER_FILE_CONSTANT_NAMES

Names of standard metadata columns that are materialised by the producer-side constant-block path (per-file values, including SQL NULL where unavailable). The other standard names (ID, SOURCE) are not in this set — they go through per-row composition operators (ExternalRowIdentity, SynthesizeExternalSource).

STANDARD_NAMES

public static final Set<String> STANDARD_NAMES

Every standard metadata name an external relation can bind — PER_FILE_CONSTANT_NAMES plus the per-row composed pair (ID, SOURCE). This is the dedicated namespace: dataset layouts (Hive partition directories, data columns) cannot claim these names; see HivePartitionDetector for the rename that enforces it.

RESERVED_NAMES

public static final Set<String> RESERVED_NAMES

The dedicated metadata namespace for reservation/rename purposes: STANDARD_NAMES plus every standard name that is only snapshot-gated for binding (_tier). Reservation must not flip with build mode — a dataset layout claiming _tier is renamed to _partition._tier in EVERY build, even where METADATA _tier itself is not yet exposed — so a Hive dataset surfaces the same column names regardless of snapshot vs release. Use this set for namespace protection; use STANDARD_NAMES for what a relation may actually bind.

Method Details

extractPerFileConstants

public static Map<String,Object> extractPerFileConstants(@Nullable
 String datasetName,
 FileList fileList,
 int index)

Build the per-file constant values for the standard metadata names listed in PER_FILE_CONSTANT_NAMES. The map is suitable for merging into a partition-value map consumed by VirtualColumnIterator. Values are:

• _index — datasetName when known, otherwise null (bare-glob FROM queries have no dataset identity).
• _version — FileList.lastModifiedMillis(int) as a Long; 0L is treated as "unknown" and yields null per the precedent set by FileMetadataColumns.extractValues(FileList, int).
• Every other standard name — null. They are not addressable on external data (no relevance scoring, no per-row _ignored list, etc.).

Callers must call this once per file; the result is meant to overlay onto the partition-value map so VirtualColumnIterator renders constant blocks of the correct type (DataType) — null values are turned into newConstantNullBlock by the iterator's existing path.

extractPerFileConstants

public static Map<String,Object> extractPerFileConstants(@Nullable
 String datasetName,
 @Nullable
 Long lastModifiedMillis)

Variant for callers that already hold the file's last-modified epoch-millis (e.g. the slice-queue path, which reuses FileMetadataColumns.MODIFIED previously stuffed into the FileSplit's partition values). A null lastModifiedMillis yields a null _version; zero is treated as "unknown" per FileMetadataColumns.extractValues(FileList, int) precedent.