Interface EvalOperator.ExpressionEvaluator

All Superinterfaces:
AutoCloseable, Closeable, org.elasticsearch.core.Releasable
All Known Implementing Classes:
LuceneQueryExpressionEvaluator
Enclosing class:
EvalOperator

public static interface EvalOperator.ExpressionEvaluator extends org.elasticsearch.core.Releasable
Evaluates an expression a + b or log(c) one Page at a time.

Eval

The primary interface is the eval(Page) method which performs the actual evaluation. Generally implementations are built in a tree structure with member EvalOperator.ExpressionEvaluator for each of their parameters. So eval(Page) will typically look like:


   Block lhs = this.lhs.eval(page);
   Block rhs = this.lhs.eval(page);
   try (Block.Builder result = ...) {
       for (int p = 0; p < lhs.getPositionCount(); p++) {
           result.add(doTheThing(lhs.get(p), rhs.get(p)));
       }
   }
 

There are hundreds of them and none of them look just like that, but that's the theory. Get Blocks from the children, then evaluate all the rows in a tight loop that hopefully can get vectorized.

Implementations need not be thread safe. A new one is built for each Driver and Drivers are only ever run in one thread at a time. Many implementations allocate "scratch" buffers for temporary memory that they reuse on each call to eval(org.elasticsearch.compute.data.Page).

Implementations must be ok with being called in by different threads, though never at the same time. It's possible that the instance belonging to a particular Driver is called on thread A many times. And then the driver yields. After a few seconds the Driver could be woken on thread B and will then call eval(Page). No two threads will ever call eval(Page) at the same time on the same instance. This rarely matters, but some implementations that interact directly with Lucene will need to check that the Thread.currentThread() is the same as the previous thread. If it isn't they'll need to reinit Lucene stuff.

Memory tracking

Implementations should track their memory usage because it's possible for us a single ESQL operation to make hundreds of them. Unlike with Accountable we have a baseRamBytesUsed() which can be read just after creation and is the sum of the ram usage of the tree of EvalOperator.ExpressionEvaluators while "empty". If an implementation much allocate any scratch memory this is not included.

baseRamBytesUsed() memory is tracked in EvalOperator. Implementation that don't allocate any scratch memory need only implement this and use DriverContext.blockFactory() to build results.

Implementations that do allocate memory should use BreakingBytesRefBuilder or BigArrays or some other safe allocation mechanism. If that isn't possible they should communicate with the CircuitBreaker directly via DriverContext.breaker().

  • Nested Class Summary

    Nested Classes
    Modifier and Type
    Interface
    Description
    static interface 
    A Factory for creating ExpressionEvaluators.
  • Method Summary

    Modifier and Type
    Method
    Description
    long
    Heap used by the evaluator excluding any memory that's separately tracked like the BreakingBytesRefBuilder used for string concat.
    eval(Page page)
    Evaluate the expression.

    Methods inherited from interface org.elasticsearch.core.Releasable

    close
  • Method Details

    • eval

      Block eval(Page page)
      Evaluate the expression.
      Returns:
      the returned Block has its own reference and the caller is responsible for releasing it.
    • baseRamBytesUsed

      long baseRamBytesUsed()
      Heap used by the evaluator excluding any memory that's separately tracked like the BreakingBytesRefBuilder used for string concat.