Package org.elasticsearch.xpack.esql.plan.logical.join

Class MarkJoin

java.lang.Object
org.elasticsearch.xpack.esql.core.tree.Node<LogicalPlan>
org.elasticsearch.xpack.esql.plan.QueryPlan<LogicalPlan>
org.elasticsearch.xpack.esql.plan.logical.LogicalPlan
org.elasticsearch.xpack.esql.plan.logical.BinaryPlan
org.elasticsearch.xpack.esql.plan.logical.join.Join
org.elasticsearch.xpack.esql.plan.logical.join.AbstractSubqueryJoin
org.elasticsearch.xpack.esql.plan.logical.join.MarkJoin
All Implemented Interfaces:
NamedWriteable, Writeable, PostAnalysisVerificationAware, PostOptimizationVerificationAware, Resolvable, ExecutesOn, ExecutesOn.Coordinator, SortAgnostic, SortPreserving
public class MarkJoin extends AbstractSubqueryJoin
A mark join used to implement field IN (subquery) when the IN is embedded in an arbitrary boolean expression — typically as a child of OR where SemiJoin / AntiJoin's row-filtering shape is not applicable.

Unlike SemiJoin, this operator preserves every left row and adds a single boolean mark attribute holding the value of field IN (subquery) for that row, with full three-valued logic:

  • TRUE — left key matches at least one row in the subquery
  • FALSE — no match, left key non-NULL, subquery had no NULLs
  • NULL — left key is NULL, OR no match but the subquery contains NULL(s)
The mark is a normal boolean attribute that the rewritten WHERE condition references, so the surrounding OR/AND/NOT operators are evaluated by the standard expression machinery.

Like SemiJoin, the right side is an independent subquery executed first; once its result arrives as a LocalRelation, AbstractSubqueryJoin.inlineData(org.elasticsearch.xpack.esql.plan.logical.join.AbstractSubqueryJoin, org.elasticsearch.xpack.esql.plan.logical.local.LocalRelation, int, org.elasticsearch.compute.data.BlockFactory, java.util.concurrent.atomic.AtomicReference<org.elasticsearch.compute.data.Page>) routes the dedup result into the buildFilterPathPlan(org.elasticsearch.compute.data.Block, org.elasticsearch.xpack.esql.core.type.DataType, org.elasticsearch.xpack.esql.core.expression.Attribute, org.elasticsearch.xpack.esql.core.tree.Source, boolean) / buildHashJoinPathPlan(org.elasticsearch.xpack.esql.plan.logical.LogicalPlan, org.elasticsearch.xpack.esql.plan.logical.local.LocalRelation, org.elasticsearch.xpack.esql.plan.logical.join.JoinConfig, org.elasticsearch.xpack.esql.core.expression.Attribute, org.elasticsearch.xpack.esql.core.tree.Source, boolean) hooks, which substitute an Eval that materializes the mark.

  • Constructor Details

  • Method Details

    • markAttribute

      public Attribute markAttribute()

    • info

      protected NodeInfo<Join> info()
      Description copied from class: Node
      Normally, you want to use one of the static create methods to implement this.

      For QueryPlans, it is very important that the properties contain all of the expressions and references relevant to this node, and that all the properties are used in the provided constructor; otherwise query plan transformations like QueryPlan#transformExpressionsOnly(Function) will not have an effect.

      Overrides:
      info in class Join

    • replaceChildren

      public Join replaceChildren(LogicalPlan left, LogicalPlan right)
      Overrides:
      replaceChildren in class Join

    • computeOutputExpressions

      public List<NamedExpression> computeOutputExpressions(List<? extends NamedExpression> left, List<? extends NamedExpression> right)
      Overrides:
      computeOutputExpressions in class AbstractSubqueryJoin

    • buildEmptyRightSidePlan

      protected LogicalPlan buildEmptyRightSidePlan(Source source)
      Description copied from class: AbstractSubqueryJoin
      Build the terminal plan when the right side has zero rows. x IN () ≡ FALSE for SEMI / MARK; x NOT IN () ≡ TRUE for ANTI. MarkJoin produces an Eval that sets the mark attribute to FALSE.
      Specified by:
      buildEmptyRightSidePlan in class AbstractSubqueryJoin

    • buildShortCircuitPlan

      protected LogicalPlan buildShortCircuitPlan(Source source, boolean allRightNull)
      Description copied from class: AbstractSubqueryJoin
      Build the terminal plan when the right side contains a NULL value that forces the predicate to a constant for every left row. For SEMI this only fires when every right value is NULL (allRightNull == true) and produces Filter(FALSE). For ANTI any NULL on the right is fatal, so it overrides AbstractSubqueryJoin.shortCircuitOnAnyRightNull() and likewise produces Filter(FALSE). MarkJoin does not short-circuit on any NULL — it keeps the row counts unchanged and emits Eval($m = NULL) when every right value is NULL.
      Overrides:
      buildShortCircuitPlan in class AbstractSubqueryJoin

    • buildFilterPathPlan

      protected LogicalPlan buildFilterPathPlan(Block dedupKeys, DataType keyType, Attribute leftField, Source source, boolean rightHadNulls)
      Description copied from class: AbstractSubqueryJoin
      Build the terminal plan for the small-dedup "filter" path. dedupKeys contains the BlockHash-deduplicated distinct values from the right side and may include a single NULL position at index 0 if any input position was NULL or multi-valued (BlockHash collapses all NULL inputs into its reserved group 0). rightHadNulls is equivalent to dedupKeys.isNull(0) and is passed through for documentation / hash-join-path symmetry. SEMI returns Filter(In(...)), ANTI returns Filter(NOT In(...)) (ANTI never reaches this path with rightHadNulls = true — it short-circuits earlier), and MarkJoin builds an Eval whose IN list is the dedup positions verbatim, relying on In's three-valued semantics to compute the mark.
      Overrides:
      buildFilterPathPlan in class AbstractSubqueryJoin

    • buildHashJoinPathPlan

      protected LogicalPlan buildHashJoinPathPlan(LogicalPlan leftSide, LocalRelation deduplicatedData, JoinConfig leftJoinConfig, Attribute sentinelAttr, Source source, boolean rightHadNulls)
      Description copied from class: AbstractSubqueryJoin
      Build the terminal plan for the large-dedup "hash-join" path. The deduplicated key column has already been wrapped in a LocalRelation alongside a constant TRUE sentinel column. SEMI/ANTI add a sentinel filter and Project that drops the right-side column; MarkJoin instead computes the mark via a CASE expression and projects the mark column out alongside the original left output.
      Overrides:
      buildHashJoinPathPlan in class AbstractSubqueryJoin

    • filterNullLeftKeysBeforeHashJoin

      protected boolean filterNullLeftKeysBeforeHashJoin()
      Description copied from class: AbstractSubqueryJoin
      Whether the hash-join path needs to drop NULL-keyed left rows before the join. SEMI/ANTI filter on the sentinel after the join; ANTI in particular would otherwise keep NULL-keyed rows (sentinel NULL → "no match") even though NULL NOT IN (...) should yield NULL. MARK keeps every left row and handles the NULL-key case explicitly inside the CASE expression that produces the mark, so it suppresses the filter.
      Overrides:
      filterNullLeftKeysBeforeHashJoin in class AbstractSubqueryJoin

    • hashCode

      public int hashCode()
      Overrides:
      hashCode in class Join

    • equals

      public boolean equals(Object obj)
      Overrides:
      equals in class Join