Interface LucenePushdownPredicates


public interface LucenePushdownPredicates
When deciding if a filter or topN can be pushed down to Lucene, we need to check a few things on the field. Exactly what is checked depends on the type of field and the query. For example, we have the following possible combinations:
  1. A normal filter on a normal field will be pushed down using SingleValueQuery to remove multi-valued results, and this requires knowing if the field is indexed and has doc-values.
  2. A filter using a spatial function will allow multi-valued fields and we only need to know if the field is indexed, and do not need doc values.
  3. A TopN will be pushed down if the field is indexed and has doc values.
  4. Filters with TEXT fields can only be pushed down if the TEXT field has a nested KEYWORD field, referred to here as ExactSubfield. This that this is related to normal ES|QL predicates, not the full-text search provided by the MATCH and QSTR functions, which are pushed down separately.
  • Field Details

    • DEFAULT

      static final LucenePushdownPredicates DEFAULT
      The default implementation of this has no access to SearchStats, so it can only make decisions based on the FieldAttribute itself. In particular, it assumes TEXT fields have no exact subfields (underlying keyword field), and that isAggregatable means indexed and has hasDocValues.
  • Method Details

    • hasExactSubfield

      boolean hasExactSubfield(FieldAttribute attr)
      For TEXT fields, we need to check if the field has a subfield of type KEYWORD that can be used instead.
    • isIndexedAndHasDocValues

      boolean isIndexedAndHasDocValues(FieldAttribute attr)
      For pushing down TopN and for pushing down filters with SingleValueQuery, we need to check if the field is indexed and has doc values.
    • isIndexed

      boolean isIndexed(FieldAttribute attr)
      For pushing down filters when multi-value results are allowed (spatial functions like ST_INTERSECTS), we only need to know if the field is indexed.
    • isPushableFieldAttribute

      default boolean isPushableFieldAttribute(Expression exp)
      We see fields as pushable if either they are aggregatable or they are indexed. This covers non-indexed cases like AbstractScriptFieldType which hard-coded isAggregatable to true, as well as normal FieldAttribute's which can only be pushed down if they are indexed. The reason we don't just rely entirely on isAggregatable is because this is often false for normal fields, and could also differ from node to node, and we can physically plan each node separately, allowing Lucene pushdown on the nodes that support it, and relying on the compute engine for the nodes that do not.
    • isPushableTextFieldAttribute

      static boolean isPushableTextFieldAttribute(Expression exp)
    • isPushableMetadataAttribute

      static boolean isPushableMetadataAttribute(Expression exp)
    • isPushableAttribute

      default boolean isPushableAttribute(Expression exp)
    • checkIsPushableAttribute

      static TypedAttribute checkIsPushableAttribute(Expression e)
    • checkIsFieldAttribute

      static FieldAttribute checkIsFieldAttribute(Expression e)
    • pushableAttributeName

      static String pushableAttributeName(TypedAttribute attribute)
    • from

      If we have access to SearchStats over a collection of shards, we can make more fine-grained decisions about what can be pushed down. This should open up more opportunities for lucene pushdown.