Package org.elasticsearch.xpack.esql.datasources.spi

Interface SplitStats

All Known Implementing Classes:
MergedSplitStats, SplitStats
public interface SplitStats
Per-split statistics used by the optimizer for aggregate pushdown and filter pruning. Implemented by SplitStats for individual file splits and by MergedSplitStats for composite splits that aggregate statistics lazily from their children.

Unknown numeric values are represented as -1; unknown object values as null. This matches the sentinel convention used throughout the datasources layer (see ExternalSplit.estimatedSizeInBytes()).

  • Method Summary

    Modifier and Type
    Method
    Description
    Object
    columnMax(String name)
    Maximum value for the named column, or null if unknown or the column is not present in this split's statistics.
    Object
    columnMin(String name)
    Minimum value for the named column, or null if unknown or the column is not present in this split's statistics.
    long
    columnNullCount(String name)
    Number of null values in the named column under the "implicit nulls" contract: a column that is physically absent from this split contributes rowCount() implicit nulls (every row would deserialize as null).
    default long
    columnSizeBytes(String name)
    Uncompressed size in bytes for the named column, or -1 if unknown or the column is not present in this split's statistics.
    default long
    columnValueCount(String name)
    Number of non-null values in the named column (multivalue-aware: a multivalued cell contributes one per value).
    default long
    compressedSizeInBytes()
    On-disk (compressed) size of this split, in bytes.
    default boolean
    hasColumn(String name)
    Whether the named column carries any per-column statistics in this split (a null count, a min/max, or a size).
    long
    rowCount()
    Total row count for this split.
    long
    sizeInBytes()
    Uncompressed size of the data in this split, in bytes.

  • Method Details

    • rowCount

      long rowCount()
      Total row count for this split. Always >= 0.

    • sizeInBytes

      long sizeInBytes()
      Uncompressed size of the data in this split, in bytes. Returns -1 if unknown.

    • compressedSizeInBytes

      default long compressedSizeInBytes()
      On-disk (compressed) size of this split, in bytes. Returns -1 if unknown.

    • columnNullCount

      long columnNullCount(String name)
      Number of null values in the named column under the "implicit nulls" contract: a column that is physically absent from this split contributes rowCount() implicit nulls (every row would deserialize as null). Returns -1 only when the column is physically present in the split but the reader could not extract a null count (the rare Parquet present-but-stats-less case; ORC always emits one).

      This contract makes Count(col) = rowCount - columnNullCount correct for UNION_BY_NAME pushdown across files where some files lack the column, and lets IS NULL/IS NOT NULL classifiers treat absent-column splits as unconditionally null without needing a separate "column present?" probe.

      Producer contract: implementations must mark a column as physically present by carrying at least one column-family stat (e.g. size_bytes, a min/max value, or a null count). Both Parquet and ORC readers satisfy this — Parquet always emits size_bytes and ORC always emits null_count. Producers that build stats by hand must follow the same rule, otherwise this method will silently report a present column as absent and inflate the implicit-null contribution.

    • columnValueCount

      default long columnValueCount(String name)
      Number of non-null values in the named column (multivalue-aware: a multivalued cell contributes one per value). This is what COUNT(col) returns, served directly rather than derived from rowCount - columnNullCount (which under-counts multivalued columns).

      Returns -1 when not available; the caller then falls back to rowCount - nullCount.

    • hasColumn

      default boolean hasColumn(String name)
      Whether the named column carries any per-column statistics in this split (a null count, a min/max, or a size). This is the "column was observed by the stats layer" predicate, distinct from columnNullCount(java.lang.String)'s implicit-nulls contract: for footer formats an absent column is genuinely all-null and columnNullCount returns rowCount, but a text-format partial harvest can leave a physically present column with no stats at all, and the two cases are indistinguishable through columnNullCount alone. Callers that must not apply the implicit-nulls contract for unharvested columns (see AggregatePushdownSupport.appliesImplicitNullsForAbsentColumn()) use this to safe-miss instead.

      Defaults to false (conservative — "no per-column stats observed"). Footer-format implementations never consult it (they declare appliesImplicitNullsForAbsentColumn), so they need not override it; partial-harvest text formats override it with the real predicate.

    • columnMin

      @Nullable Object columnMin(String name)
      Minimum value for the named column, or null if unknown or the column is not present in this split's statistics.

    • columnMax

      @Nullable Object columnMax(String name)
      Maximum value for the named column, or null if unknown or the column is not present in this split's statistics.

    • columnSizeBytes

      default long columnSizeBytes(String name)
      Uncompressed size in bytes for the named column, or -1 if unknown or the column is not present in this split's statistics.