Package org.elasticsearch.xpack.esql.datasources.pushdown

Class PushdownPredicates

java.lang.Object
org.elasticsearch.xpack.esql.datasources.pushdown.PushdownPredicates
public final class PushdownPredicates extends Object
Shared structural validation for predicate pushdown across file formats (ORC, Parquet, etc.).

These methods check whether an ESQL expression has the right structure for pushdown: field reference on the left, foldable literal on the right, supported operator, and a data type the target format can handle. The format-specific type check is provided as a Predicate<DataType> so each format plugs in its own supported types.

Boolean connective rules (AND partial vs full pushdown, OR, NOT) are intentionally excluded — ORC allows partial AND pushdown while Iceberg requires both sides, so the traversal logic stays format-specific.

Virtual columns (MetadataAttribute for ES-index metadata like _id/_index, any VirtualAttribute for engine-synthesized columns like _file.*) never live in the physical file schema, so format-level readers cannot evaluate predicates / aggregates against them and these helpers always reject them. Identification is type-based: name conventions (e.g. the _file. prefix) are surface and may be aliased away, but the marker interface stays attached across renames and serialization.

  • Method Summary

    Modifier and Type
    Method
    Description
    static boolean
    isComparison(EsqlBinaryComparison bc, Predicate<DataType> typeSupported)
    Checks if a binary comparison can be pushed down: left side is a non-virtual field reference, right side is a foldable literal, operator is one of the six standard comparisons, and the field's data type is supported by the target format.
    static boolean
    isContains(Contains c, Predicate<DataType> typeSupported)
    Checks if a Contains expression can be pushed down: field is a single-value named expression with a supported data type, and the substring is foldable.
    static boolean
    isEndsWith(EndsWith ew, Predicate<DataType> typeSupported)
    Checks if an EndsWith expression can be pushed down: field is a single-value named expression with a supported data type, and the suffix is foldable.
    static boolean
    isIn(In inExpr, Predicate<DataType> typeSupported)
    Checks if an IN expression can be pushed down: value is a non-virtual field reference with a supported data type, all list items are foldable, and at least one is non-null.
    static boolean
    isIsNotNull(IsNotNull isNotNull, Predicate<DataType> typeSupported)
    Checks if an IS NOT NULL expression can be pushed down: field is a non-virtual named expression with a supported data type.
    static boolean
    isIsNull(IsNull isNull, Predicate<DataType> typeSupported)
    Checks if an IS NULL expression can be pushed down: field is a non-virtual named expression with a supported data type.
    static boolean
    isRange(Range range, Predicate<DataType> typeSupported)
    Checks if a range expression can be pushed down: value is a non-virtual field reference with a supported data type, and both bounds are foldable.
    static boolean
    isStartsWith(StartsWith sw, Predicate<DataType> typeSupported)
    Checks if a StartsWith expression can be pushed down: field is a single-value named expression with a supported data type, and the prefix is foldable.
    static boolean
    isVirtualColumn(Expression e)
    Returns true when e represents a column that has no physical presence in the source data and therefore cannot be evaluated by a format-level scan.

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Method Details

    • isVirtualColumn

      public static boolean isVirtualColumn(Expression e)
      Returns true when e represents a column that has no physical presence in the source data and therefore cannot be evaluated by a format-level scan. Two cases:
      • MetadataAttribute — Elasticsearch document metadata (_id, _index, _score, ...). Real per-document values, but external-file readers (Parquet, ORC) have no concept of them.
      • Any VirtualAttribute — engine-synthesized columns (_file.* today) materialized by VirtualColumnIterator on the producer thread.
      Format-level filter and aggregate pushdown rules must reject both. Use this helper rather than name- or prefix-based checks: the marker survives rename/aliasing/serialization, names do not.

    • isComparison

      public static boolean isComparison(EsqlBinaryComparison bc, Predicate<DataType> typeSupported)
      Checks if a binary comparison can be pushed down: left side is a non-virtual field reference, right side is a foldable literal, operator is one of the six standard comparisons, and the field's data type is supported by the target format.

    • isIn

      public static boolean isIn(In inExpr, Predicate<DataType> typeSupported)
      Checks if an IN expression can be pushed down: value is a non-virtual field reference with a supported data type, all list items are foldable, and at least one is non-null.

    • isIsNull

      public static boolean isIsNull(IsNull isNull, Predicate<DataType> typeSupported)
      Checks if an IS NULL expression can be pushed down: field is a non-virtual named expression with a supported data type.

    • isIsNotNull

      public static boolean isIsNotNull(IsNotNull isNotNull, Predicate<DataType> typeSupported)
      Checks if an IS NOT NULL expression can be pushed down: field is a non-virtual named expression with a supported data type.

    • isRange

      public static boolean isRange(Range range, Predicate<DataType> typeSupported)
      Checks if a range expression can be pushed down: value is a non-virtual field reference with a supported data type, and both bounds are foldable.

    • isStartsWith

      public static boolean isStartsWith(StartsWith sw, Predicate<DataType> typeSupported)
      Checks if a StartsWith expression can be pushed down: field is a single-value named expression with a supported data type, and the prefix is foldable.

    • isEndsWith

      public static boolean isEndsWith(EndsWith ew, Predicate<DataType> typeSupported)
      Checks if an EndsWith expression can be pushed down: field is a single-value named expression with a supported data type, and the suffix is foldable.

    • isContains

      public static boolean isContains(Contains c, Predicate<DataType> typeSupported)
      Checks if a Contains expression can be pushed down: field is a single-value named expression with a supported data type, and the substring is foldable.