Interface LucenePushdownPredicates
public interface LucenePushdownPredicates
When deciding if a filter or topN can be pushed down to Lucene, we need to check a few things on the field.
Exactly what is checked depends on the type of field and the query. For example, we have the following possible combinations:
- A normal filter on a normal field will be pushed down using SingleValueQuery to remove multi-valued results, and this requires knowing if the field is indexed and has doc-values.
- A filter using a spatial function will allow multi-valued fields and we only need to know if the field is indexed, and do not need doc values.
- A TopN will be pushed down if the field is indexed and has doc values.
- Filters with TEXT fields can only be pushed down if the TEXT field has a nested KEYWORD field, referred to here as ExactSubfield. This that this is related to normal ES|QL predicates, not the full-text search provided by the MATCH and QSTR functions, which are pushed down separately.
-
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final LucenePushdownPredicatesThe default implementation of this has no access to SearchStats, so it can only make decisions based on the FieldAttribute itself. -
Method Summary
Modifier and TypeMethodDescriptionbooleancanUseEqualityOnSyntheticSourceDelegate(FieldAttribute attr, String value) static FieldAttributestatic TypedAttributeflags()static LucenePushdownPredicatesforCanMatch(TransportVersion minTransportVersion, EsqlFlags flags) ALucenePushdownPredicatesfor use with thecan_matchphase.static LucenePushdownPredicatesfrom(SearchStats stats, EsqlFlags flags) If we have access toSearchStatsover a collection of shards, we can make more fine-grained decisions about what can be pushed down.booleanFor TEXT fields, we need to check if the field has a subfield of type KEYWORD that can be used instead.booleanisIndexed(FieldAttribute attr) For pushing down filters when multi-value results are allowed (spatial functions like ST_INTERSECTS), we only need to know if the field is indexed.booleanFor pushing down TopN and for pushing down filters with SingleValueQuery, we need to check if the field is indexed and has doc values.default booleandefault booleanWe see fields as pushable if either they are aggregatable or they are indexed.static booleanstatic booleanIf we're extracting a query forcan_matchthen this is the minimum transport version in the cluster.static StringpushableAttributeName(TypedAttribute attribute)
-
Field Details
-
DEFAULT
The default implementation of this has no access to SearchStats, so it can only make decisions based on the FieldAttribute itself. In particular, it assumes TEXT fields have no exact subfields (underlying keyword field), and that isAggregatable means indexed and has hasDocValues.
-
-
Method Details
-
minTransportVersion
If we're extracting a query forcan_matchthen this is the minimum transport version in the cluster. Otherwise, this isnull.If this is not null
Expressions should not claim to be serializable unless theirQueryBuildersupportsthe version.This is done on the coordinating node and. And for cross cluster search this is done on the coordinating node on the remote cluster. So! We actually have the minimum cluster transport version.
-
flags
EsqlFlags flags() -
hasExactSubfield
For TEXT fields, we need to check if the field has a subfield of type KEYWORD that can be used instead. -
isIndexedAndHasDocValues
For pushing down TopN and for pushing down filters with SingleValueQuery, we need to check if the field is indexed and has doc values. -
isIndexed
For pushing down filters when multi-value results are allowed (spatial functions like ST_INTERSECTS), we only need to know if the field is indexed. -
canUseEqualityOnSyntheticSourceDelegate
-
isPushableFieldAttribute
We see fields as pushable if either they are aggregatable or they are indexed. This covers non-indexed cases likeAbstractScriptFieldTypewhich hard-codedisAggregatableto true, as well as normalFieldAttribute's which can only be pushed down if they are indexed. The reason we don't just rely entirely onisAggregatableis because this is often false for normal fields, and could also differ from node to node, and we can physically plan each node separately, allowing Lucene pushdown on the nodes that support it, and relying on the compute engine for the nodes that do not. -
isPushableTextFieldAttribute
-
isPushableMetadataAttribute
-
isPushableAttribute
-
checkIsPushableAttribute
-
checkIsFieldAttribute
-
pushableAttributeName
-
forCanMatch
ALucenePushdownPredicatesfor use with thecan_matchphase. -
from
If we have access toSearchStatsover a collection of shards, we can make more fine-grained decisions about what can be pushed down. This should open up more opportunities for lucene pushdown.
-