Module org.elasticsearch.compute
Package org.elasticsearch.compute.data.arrow

Class BytesRefArrowBlock

java.lang.Object
org.elasticsearch.compute.data.arrow.BytesRefArrowBlock
public final class BytesRefArrowBlock extends Object
Converts Arrow VARCHAR/VARBINARY vectors to ESQL BytesRefBlocks. Flat vectors delegate to zero-copy BytesRefArrowBufBlock. ListVector (multi-valued) inputs are converted by copying values into block builders, because BytesRefArrowBufBlock.expand() does not handle the variable-width value offsets needed by downstream operators like MvExpand. See ArrowListSupport for the Arrow list -> ESQL multi-value mapping rules (null lists, empty lists, and null children are all collapsed to an ESQL null position; mixed lists drop their null elements).

  • Method Details

    • of

      public static Block of(org.apache.arrow.vector.ValueVector vector, BlockFactory blockFactory)

    • ofSanitizedUtf8

      public static Block ofSanitizedUtf8(org.apache.arrow.vector.ValueVector vector, BlockFactory blockFactory)
      Converts an Arrow VARCHAR vector to a KEYWORD BytesRefBlock, repairing malformed UTF-8 to U+FFFD so downstream KEYWORD operations (e.g. the TopN Utf8 encoders) stay total. Arrow VARCHAR is nominally UTF-8, but external producers (e.g. Spark) can still emit ill-formed bytes.

      The common, well-formed case is validated in a single pass and then returned as the zero-copy BytesRefArrowBufBlock; only when a malformed value is found does the block get rebuilt with sanitized values. UTF-8 well-formedness is checked per value because a multi-byte sequence can never span two logical Arrow values.