Interface LucenePushdownPredicates
public interface LucenePushdownPredicates
When deciding if a filter or topN can be pushed down to Lucene, we need to check a few things on the field.
Exactly what is checked depends on the type of field and the query. For example, we have the following possible combinations:
- A normal filter on a normal field will be pushed down using SingleValueQuery to remove multi-valued results, and this requires knowing if the field is indexed and has doc-values.
- A filter using a spatial function will allow multi-valued fields and we only need to know if the field is indexed, and do not need doc values.
- A TopN will be pushed down if the field is indexed and has doc values.
- Filters with TEXT fields can only be pushed down if the TEXT field has a nested KEYWORD field, referred to here as ExactSubfield. This that this is related to normal ES|QL predicates, not the full-text search provided by the MATCH and QSTR functions, which are pushed down separately.
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final LucenePushdownPredicatesThe default implementation of this has no access to SearchStats, so it can only make decisions based on the FieldAttribute itself. -
Method Summary
Modifier and TypeMethodDescriptionstatic FieldAttributestatic TypedAttributestatic LucenePushdownPredicatesfrom(SearchStats stats) If we have access toSearchStatsover a collection of shards, we can make more fine-grained decisions about what can be pushed down.booleanFor TEXT fields, we need to check if the field has a subfield of type KEYWORD that can be used instead.booleanisIndexed(FieldAttribute attr) For pushing down filters when multi-value results are allowed (spatial functions like ST_INTERSECTS), we only need to know if the field is indexed.booleanFor pushing down TopN and for pushing down filters with SingleValueQuery, we need to check if the field is indexed and has doc values.default booleandefault booleanWe see fields as pushable if either they are aggregatable or they are indexed.static booleanstatic booleanstatic StringpushableAttributeName(TypedAttribute attribute)
-
Field Details
-
DEFAULT
The default implementation of this has no access to SearchStats, so it can only make decisions based on the FieldAttribute itself. In particular, it assumes TEXT fields have no exact subfields (underlying keyword field), and that isAggregatable means indexed and has hasDocValues.
-
-
Method Details
-
hasExactSubfield
For TEXT fields, we need to check if the field has a subfield of type KEYWORD that can be used instead. -
isIndexedAndHasDocValues
For pushing down TopN and for pushing down filters with SingleValueQuery, we need to check if the field is indexed and has doc values. -
isIndexed
For pushing down filters when multi-value results are allowed (spatial functions like ST_INTERSECTS), we only need to know if the field is indexed. -
isPushableFieldAttribute
We see fields as pushable if either they are aggregatable or they are indexed. This covers non-indexed cases likeAbstractScriptFieldTypewhich hard-codedisAggregatableto true, as well as normalFieldAttribute's which can only be pushed down if they are indexed. The reason we don't just rely entirely onisAggregatableis because this is often false for normal fields, and could also differ from node to node, and we can physically plan each node separately, allowing Lucene pushdown on the nodes that support it, and relying on the compute engine for the nodes that do not. -
isPushableTextFieldAttribute
-
isPushableMetadataAttribute
-
isPushableAttribute
-
checkIsPushableAttribute
-
checkIsFieldAttribute
-
pushableAttributeName
-
from
If we have access toSearchStatsover a collection of shards, we can make more fine-grained decisions about what can be pushed down. This should open up more opportunities for lucene pushdown.
-