Package org.elasticsearch.xpack.esql.datasources

Class ExternalSchema

java.lang.Object
org.elasticsearch.xpack.esql.datasources.ExternalSchema
All Implemented Interfaces:
Iterable<Attribute>
public final class ExternalSchema extends Object implements Iterable<Attribute>
Coordinator-only carrier for the four-schema-model units (File, Unified, Query, Per-file Query — see SchemaReconciliation) inside the datasources subsystem. Pairs a positional List<Attribute> with a name set built eagerly at construction for O(1) membership tests at the prune / skip / map-filter seams. Not a wire form — schemas that travel across nodes go through their own paths (post-prune attributes on ExternalSourceExec via the planner contract, and FileSplit.readSchema via primitive (name, type, nullable) tuples).

  • Field Details

  • Constructor Details

    • ExternalSchema

      public ExternalSchema(List<Attribute> attributes)

  • Method Details

    • dataAttributesOf

      public static ExternalSchema dataAttributesOf(List<Attribute> attributes)
      Returns the data-attribute view of attributes as an ExternalSchema: the input list with virtual columns filtered out (relative order preserved). Equivalent to dataAttributesOf(List, Set) with an empty partition-column set; callers that track Hive-style partition columns must use that overload so the resulting width agrees with the file-backed ColumnMapping.

    • dataAttributesOf

      public static ExternalSchema dataAttributesOf(List<Attribute> attributes, Set<String> partitionColumnNames)
      Returns the data-attribute view of attributes as an ExternalSchema: the input list with virtual columns and Hive-style partition columns filtered out (relative order preserved). Used by external-source operator factories on the data node to derive the data-only schema once at construction rather than re-slicing per page.

      Two classes of column are excluded because the format reader never produces them in the data channel — both are appended on the producer thread by VirtualColumnIterator:

      • Virtual columns (MetadataAttribute for ES document metadata, any VirtualAttribute for engine-synthesized columns like _file.*).
      • Hive-style partition columns — real ReferenceAttributes whose name is in partitionColumnNames. Their values are derived from the file path, not the file body. Excluding them keeps the data-only width in agreement with the file-backed ColumnMapping; otherwise the SchemaAdaptingIterator size-vs-width guard misfires whenever a partition key shadows a same-named physical column (the path-derived value wins and the physical column is hidden, matching Spark and DuckDB).

    • attributes

      public List<Attribute> attributes()

    • names

      public Set<String> names()

    • size

      public int size()

    • isEmpty

      public boolean isEmpty()

    • get

      public Attribute get(int index)

    • iterator

      public Iterator<Attribute> iterator()
      Specified by:
      iterator in interface Iterable<Attribute>

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object