Package org.elasticsearch.xpack.esql.datasources

Class SplitStats

java.lang.Object
org.elasticsearch.xpack.esql.datasources.SplitStats
All Implemented Interfaces:
Writeable, SplitStats
public final class SplitStats extends Object implements SplitStats, Writeable
Compact representation of per-split (row-group/stripe) statistics with path-segment dictionary compression. Replaces raw Map<String, Object> for wire efficiency: column names are stored as ordinal references into a shared segment dictionary, and per-column stats use parallel arrays instead of string-keyed map entries.

Use SplitStats.Builder to construct instances from format reader metadata, and toMap() for backward-compatible access by consumers that expect the flat _stats.* key convention from SourceStatisticsSerializer.

Instances are effectively immutable after construction and safe for use within a single request processing thread.

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Class
    Description
    static final class 
    SplitStats.Builder
    Builder for constructing SplitStats instances.

    Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

    Writeable.Reader<V>, Writeable.Writer<V>

  • Field Summary

    Fields
    Modifier and Type
    Field
    Description
    static final SplitStats
    EMPTY
    Empty stats with zero rows and no columns.

  • Constructor Summary

    Constructors
    Constructor
    Description
    SplitStats(StreamInput in)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    int
    columnCount()
     
    Object
    columnMax(String name)
    Returns the max value for the column with the given name, or null if not found or poisoned (unservable).
    Object
    columnMin(String name)
    Returns the min value for the column with the given name, or null if not found or poisoned (unservable).
    String
    columnName(int ordinal)
    Resolves the full dotted column name for the given ordinal by joining the path segments from the dictionary.
    long
    columnNullCount(String name)
    Returns the null count for the column with the given name under the SPI's "implicit nulls" contract: a column physically absent from this split contributes rowCount implicit nulls, since every row would deserialize as null.
    long
    columnSizeBytes(String name)
    Returns the size in bytes for the column with the given name, or -1 if not found.
    long
    columnValueCount(String name)
    Returns the count of non-null values for the column with the given name (multivalue-aware: a multivalued cell contributes one per value), or -1 if the column is not present in this split's statistics.
    boolean
    equals(Object o)
     
    int
    findColumn(String name)
    Finds a column ordinal by its full dotted name.
    boolean
    hasColumn(String name)
    Returns true iff this split carries a column-family entry for name.
    int
    hashCode()
     
    Object
    max(int col)
    Returns the max value for the column, or null if unknown or poisoned (unservable).
    Object
    min(int col)
    Returns the min value for the column, or null if unknown or poisoned (unservable).
    long
    nullCount(int col)
    Returns null count for the column, or -1 if unknown.
    static SplitStats
    of(Map<String,Object> map)
    Parses a flat _stats.* map (old format) into a SplitStats instance.
    static SplitStats
    resolveEffectiveStats(List<? extends ExternalSplit> splits, Map<String,Object> sourceMetadata)
    Resolves the effective SplitStats for a set of splits.
    long
    rowCount()
    Total row count for this split.
    long
    sizeBytes(int col)
    Returns size in bytes for the column, or -1 if unknown.
    long
    sizeInBytes()
    Returns size in bytes, or -1 if unknown.
    Map<String,Object>
    toMap()
    Produces a flat Map<String, Object> using the same _stats.* key conventions as SourceStatisticsSerializer, for backward compatibility with all existing consumers.
    String
    toString()
     
    long
    valueCount(int col)
    Returns value count (non-null values, multivalue-aware) for the column, or -1 if unknown.
    void
    writeTo(StreamOutput out)
     

    Methods inherited from class java.lang.Object

    clone, finalize, getClass, notify, notifyAll, wait, wait, wait

    Methods inherited from interface org.elasticsearch.xpack.esql.datasources.spi.SplitStats

    compressedSizeInBytes

  • Field Details

    • EMPTY

      public static final SplitStats EMPTY
      Empty stats with zero rows and no columns.

  • Constructor Details

  • Method Details

    • writeTo

      public void writeTo(StreamOutput out) throws IOException
      Specified by:
      writeTo in interface Writeable
      Throws:
      IOException

    • columnCount

      public int columnCount()

    • rowCount

      public long rowCount()
      Description copied from interface: SplitStats
      Total row count for this split. Always >= 0.
      Specified by:
      rowCount in interface SplitStats

    • sizeInBytes

      public long sizeInBytes()
      Returns size in bytes, or -1 if unknown.
      Specified by:
      sizeInBytes in interface SplitStats

    • columnName

      public String columnName(int ordinal)
      Resolves the full dotted column name for the given ordinal by joining the path segments from the dictionary. Each call allocates a new String; callers that iterate over columns should prefer resolveColumnName(int, StringBuilder) to reuse a buffer.

    • nullCount

      public long nullCount(int col)
      Returns null count for the column, or -1 if unknown.

    • valueCount

      public long valueCount(int col)
      Returns value count (non-null values, multivalue-aware) for the column, or -1 if unknown.

    • min

      @Nullable public Object min(int col)
      Returns the min value for the column, or null if unknown or poisoned (unservable).

    • max

      @Nullable public Object max(int col)
      Returns the max value for the column, or null if unknown or poisoned (unservable).

    • sizeBytes

      public long sizeBytes(int col)
      Returns size in bytes for the column, or -1 if unknown.

    • findColumn

      public int findColumn(String name)
      Finds a column ordinal by its full dotted name. Returns -1 if not found. Matches directly against the segment dictionary without materializing column name strings, keeping the compact representation intact.

    • columnNullCount

      public long columnNullCount(String name)
      Returns the null count for the column with the given name under the SPI's "implicit nulls" contract: a column physically absent from this split contributes rowCount implicit nulls, since every row would deserialize as null. Format readers (Parquet, ORC) emit at least one column-family stat key (e.g. size_bytes/null_count) for every column they physically contain, so findColumn(java.lang.String) returning -1 is equivalent to "column is not in this file/split".

      Returns -1 only in the rare case where the column is physically present but the reader could not extract a null count (e.g. Parquet stats disabled at write time). Callers such as PushStatsToExternalSource rely on this contract so that Count(col) = rowCount - columnNullCount is correct across UNION_BY_NAME mixes where some files lack the column.

      Specified by:
      columnNullCount in interface SplitStats

    • columnValueCount

      public long columnValueCount(String name)
      Returns the count of non-null values for the column with the given name (multivalue-aware: a multivalued cell contributes one per value), or -1 if the column is not present in this split's statistics. -1 signals "not available" so the caller falls back to rowCount - columnNullCount.
      Specified by:
      columnValueCount in interface SplitStats

    • hasColumn

      public boolean hasColumn(String name)
      Returns true iff this split carries a column-family entry for name. A column is present in the compact representation only when the reader emitted at least one stat for it (the producer contract documented on SplitStats.columnNullCount(java.lang.String)), so this is exactly "the stats layer observed this column" — the predicate the optimizer uses to tell a harvested-but-all-null text column apart from an unharvested one.
      Specified by:
      hasColumn in interface SplitStats

    • columnMin

      @Nullable public Object columnMin(String name)
      Returns the min value for the column with the given name, or null if not found or poisoned (unservable).
      Specified by:
      columnMin in interface SplitStats

    • columnMax

      @Nullable public Object columnMax(String name)
      Returns the max value for the column with the given name, or null if not found or poisoned (unservable).
      Specified by:
      columnMax in interface SplitStats

    • columnSizeBytes

      public long columnSizeBytes(String name)
      Returns the size in bytes for the column with the given name, or -1 if not found.
      Specified by:
      columnSizeBytes in interface SplitStats

    • toMap

      public Map<String,Object> toMap()
      Produces a flat Map<String, Object> using the same _stats.* key conventions as SourceStatisticsSerializer, for backward compatibility with all existing consumers.

    • of

      @Nullable public static SplitStats of(Map<String,Object> map)
      Parses a flat _stats.* map (old format) into a SplitStats instance. Returns null if the map is null, empty, or missing the required row count key.

      Column ordering in the result depends on the input map's iteration order: stable (e.g. LinkedHashMap) inputs produce stable ordinals. The toMap() output is always key-equivalent regardless of column ordering.

    • resolveEffectiveStats

      @Nullable public static SplitStats resolveEffectiveStats(List<? extends ExternalSplit> splits, Map<String,Object> sourceMetadata)
      Resolves the effective SplitStats for a set of splits. Uses ExternalSplit.splitStats() on each split, which handles both FileSplit and CoalescedSplit transparently without flattening. For single-split queries the per-split stats (if available) or the sourceMetadata map is used; for multi-split queries a MergedSplitStats over the per-split stats is returned. Returns null if any split lacks stats.

    • equals

      public boolean equals(Object o)
      Overrides:
      equals in class Object

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Object

    • toString

      public String toString()
      Overrides:
      toString in class Object