Package org.elasticsearch.xpack.esql.optimizer.rules.physical.local

Class ExternalSourceAggregatePushdown

java.lang.Object
org.elasticsearch.xpack.esql.optimizer.rules.physical.local.ExternalSourceAggregatePushdown
public final class ExternalSourceAggregatePushdown extends Object
Shared helpers for the aggregate pushdown rule (PushStatsToExternalSource) and the split-discovery gate (ComputeService.canSkipForAggregateOverExternal) that extract an ExternalSourceExec from the plan tree and resolve filtered metadata using SplitFilterClassifier.

  • Method Details

    • extractAggregateFunctions

      public static List<Expression> extractAggregateFunctions(List<? extends Expression> aggregates)
      Unwraps the AggregateFunction operands of an ungrouped aggregate's output, dropping any non-aggregate expressions (e.g. bare literals). This is the input every format's AggregatePushdownSupport.canPushAggregates(java.util.List<org.elasticsearch.xpack.esql.core.expression.Expression>, java.util.List<org.elasticsearch.xpack.esql.core.expression.Expression>) type-gate consumes — shared by PushStatsToExternalSource (fold) and ComputeService.canSkipForAggregateOverExternal (split-discovery gate) so both feed the gate the same function list.

    • canServeAllFromStats

      public static boolean canServeAllFromStats(List<? extends NamedExpression> aggregates, SplitStats stats, boolean implicitNullsForAbsentColumn, Set<String> pathDerivedColumns)
      Whether EVERY aggregate in aggregates would resolve from stats — the boolean twin of PushStatsToExternalSource's value-collecting loop. The split-discovery gate (ComputeService.canSkipSplitDiscovery) must consult THIS check, not just the type-level AggregatePushdownSupport.canPushAggregates(java.util.List<org.elasticsearch.xpack.esql.core.expression.Expression>, java.util.List<org.elasticsearch.xpack.esql.core.expression.Expression>): the two previously diverged on per-column servability, so the gate would skip discovery (leaving a zero-split scan) for an aggregate the fold then safe-missed, and the scan's un-pruned union_by_name mapping tripped SchemaAdaptingIterator's width guard (the union_by_name zero-split servability guard). Sharing resolveFromStats with the fold guarantees "gate skips" implies "fold serves". The bail conditions here must stay identical to PushStatsToExternalSource's value loop.

    • partitionColumnNames

      public static Set<String> partitionColumnNames(Map<String,Object> sourceMetadata)
      The columns whose values are derived from the file's directory PATH (Hive-style partition keys), not its payload. They are absent from every file's column stats, so the implicit-nulls contract reads them as all-null — any COUNT over one would serve 0. Both the fold and the split-discovery gate feed this set to resolveFromStats(org.elasticsearch.xpack.esql.core.expression.Expression, org.elasticsearch.xpack.esql.datasources.spi.SplitStats, boolean, java.util.Set<java.lang.String>) so a partition-column aggregate safe-misses on footer formats.

      Read from the SERIALIZED sourceMetadata (stamped at resolution — see SourceStatisticsSerializer#PARTITION_COLUMNS_KEY), NOT the FileList: the fileList that carries PartitionMetadata is coordinator-only and deserializes to UNRESOLVED on a data node, so the data-node fold would otherwise see an empty set and fold COUNT(partition_col) to 0. Empty when the source is not partitioned (no hive_partitioning).

    • findExternalSource

      public static ExternalSourceExec findExternalSource(PhysicalPlan child)
      Light-weight projection of extractExternalSource(PhysicalPlan) that returns just the ExternalSourceExec (or null) for callers that don't need the alias map or filter condition. Cross-package callers (the planner, other optimizer rules) use this so they share the same set of recognized wrapper shapes — adding a new shape here automatically propagates.