All Superinterfaces:
org.apache.lucene.util.Accountable, AutoCloseable, BlockLoader.Block, Closeable, org.elasticsearch.core.RefCounted, org.elasticsearch.core.Releasable, Writeable
All Known Subinterfaces:
AggregateMetricDoubleBlock, BooleanBlock, BytesRefBlock, DoubleBlock, FloatBlock, IntBlock, LongBlock
All Known Implementing Classes:
AggregateMetricDoubleArrayBlock, BooleanArrayBlock, BooleanBigArrayBlock, BooleanVectorBlock, BytesRefArrayBlock, BytesRefVectorBlock, CompositeBlock, ConstantNullBlock, DocBlock, DoubleArrayBlock, DoubleBigArrayBlock, DoubleVectorBlock, FloatArrayBlock, FloatBigArrayBlock, FloatVectorBlock, IntArrayBlock, IntBigArrayBlock, IntVectorBlock, LongArrayBlock, LongBigArrayBlock, LongVectorBlock, OrdinalBytesRefBlock

public interface Block extends org.apache.lucene.util.Accountable, BlockLoader.Block, Writeable, org.elasticsearch.core.RefCounted, org.elasticsearch.core.Releasable
A Block is a columnar representation of homogenous data. It has a position (row) count, and various data retrieval methods for accessing the underlying data that is stored at a given position.

Blocks can represent various shapes of underlying data. A Block can represent either sparse or dense data. A Block can represent either single or multivalued data. A Block that represents dense single-valued data can be viewed as a Vector.

Blocks are reference counted; to make a shallow copy of a block (e.g. if a Page contains the same column twice), use RefCounted.incRef(). Before a block is garbage collected, Releasable.close() must be called to release a block's resources; it must also be called one additional time for each time RefCounted.incRef() was called. Calls to RefCounted.decRef() and Releasable.close() are equivalent.

Block are immutable and can be passed between threads as long as no two threads hold a reference to the same block at the same time.

  • Field Details

    • MAX_LOOKUP

      static final long MAX_LOOKUP
      The maximum number of values that can be added to one position via lookup. TODO maybe make this everywhere?
      See Also:
    • PAGE_MEM_OVERHEAD_PER_BLOCK

      static final int PAGE_MEM_OVERHEAD_PER_BLOCK
      We do not track memory for pages directly (only for single blocks), but the page memory overhead can still be significant, especially for pages containing thousands of blocks. For now, we approximate this overhead, per block, using this value. The exact overhead per block would be (more correctly) RamUsageEstimator.NUM_BYTES_OBJECT_REF, but we approximate it with RamUsageEstimator.NUM_BYTES_OBJECT_ALIGNMENT to avoid further alignments to object size (at the end of the alignment, it would make no practical difference).
    • SERIALIZE_BLOCK_VALUES

      static final byte SERIALIZE_BLOCK_VALUES
      Serialization type for blocks: 0 and 1 replace false/true used in pre-8.14
      See Also:
    • SERIALIZE_BLOCK_VECTOR

      static final byte SERIALIZE_BLOCK_VECTOR
      See Also:
    • SERIALIZE_BLOCK_ARRAY

      static final byte SERIALIZE_BLOCK_ARRAY
      See Also:
    • SERIALIZE_BLOCK_BIG_ARRAY

      static final byte SERIALIZE_BLOCK_BIG_ARRAY
      See Also:
    • SERIALIZE_BLOCK_ORDINAL

      static final byte SERIALIZE_BLOCK_ORDINAL
      See Also:
  • Method Details

    • asVector

      Vector asVector()
      Returns an efficient dense single-value view of this block.. Null, if the block is not dense single-valued. That is, if mayHaveNulls returns true, or getTotalValueCount is not equal to getPositionCount.
      Returns:
      an efficient dense single-value view of this block
    • getTotalValueCount

      int getTotalValueCount()
      Returns The total number of values in this block not counting nulls..
      Returns:
      The total number of values in this block not counting nulls.
    • getPositionCount

      int getPositionCount()
      Returns The number of positions in this block..
      Returns:
      The number of positions in this block.
    • getFirstValueIndex

      int getFirstValueIndex(int position)
      Gets the index of the first value for the given position.
    • getValueCount

      int getValueCount(int position)
      Gets the number of values for the given position, possibly 0.
    • elementType

      ElementType elementType()
      Returns the element type of this block.
      Returns:
      the element type of this block
    • blockFactory

      BlockFactory blockFactory()
      The block factory associated with this block.
    • allowPassingToDifferentDriver

      void allowPassingToDifferentDriver()
      Before passing a Block to another Driver, it is necessary to switch the owning block factory to its parent, which is associated with the global circuit breaker. This ensures that when the new driver releases this Block, it returns memory directly to the parent block factory instead of the local block factory of this Block. This is important because the local block factory is not thread safe and doesn't support simultaneous access by more than one thread.
    • isReleased

      boolean isReleased()
      Tells if this block has been released. A block is released by calling its Releasable.close() or RefCounted.decRef() methods.
      Returns:
      true iff the block's reference count is zero.
    • isNull

      boolean isNull(int position)
      Parameters:
      position - the position
      Returns:
      true if the value stored at the given position is null, false otherwise
    • mayHaveNulls

      boolean mayHaveNulls()
      Returns:
      true if some values might be null. False, if all values are guaranteed to be not null.
    • areAllValuesNull

      boolean areAllValuesNull()
      Returns:
      true if all values in this block are guaranteed to be null.
    • mayHaveMultivaluedFields

      boolean mayHaveMultivaluedFields()
      Can this block have multivalued fields? Blocks that return false will never return more than one from getValueCount(int). This may return true for Blocks that do not have multivalued fields, but it will always answer quickly.
    • doesHaveMultivaluedFields

      boolean doesHaveMultivaluedFields()
      Does this block have multivalued fields? Unlike mayHaveMultivaluedFields() this will never return a false positive. In other words, if this returns true then there are positions for which getValueCount(int) will return more than 1. This will answer quickly if it can but may have to check all positions.
    • filter

      Block filter(int... positions)
      Creates a new block that only exposes the positions provided.
      Parameters:
      positions - the positions to retain
      Returns:
      a filtered block TODO: pass BlockFactory
    • keepMask

      Block keepMask(BooleanVector mask)
      Build a Block with the same values as this Block, but replacing all values for which mask.getBooleanValue(position) returns false with null. The mask vector must be at least as long as this Block.
    • lookup

      org.elasticsearch.core.ReleasableIterator<? extends Block> lookup(IntBlock positions, ByteSizeValue targetBlockSize)
      Builds an Iterator of new Blocks with the same elementType() as this Block whose values are copied from positions in this Block. It has the same number of positions as the positions parameter.

      For example, if this block contained [a, b, [b, c]] and were called with the block [0, 1, 1, [1, 2]] then the result would be [a, b, b, [b, b, c]].

      This process produces count(this) * count(positions) values per positions which could be quite large. Instead of returning a single Block, this returns an Iterator of Blocks containing all of the promised values.

      The returned ReleasableIterator may retain a reference to the positions parameter. Close it to release those references.

      This block is built using the same BlockFactory as was used to build the positions parameter.

    • mvOrdering

      Block.MvOrdering mvOrdering()
      How are multivalued fields ordered?
    • mvDeduplicated

      default boolean mvDeduplicated()
      Are multivalued fields de-duplicated in each position
    • mvSortedAscending

      default boolean mvSortedAscending()
      Are multivalued fields sorted ascending in each position
    • expand

      Block expand()
      Expand multivalued fields into one row per value. Returns the same block if there aren't any multivalued fields to expand. The returned block needs to be closed by the caller to release the block's resources.
    • insertNulls

      default Block insertNulls(IntVector before)
      Build a Block with a null inserted before each listed position.

      Note: before must be non-decreasing.

    • writeTo

      void writeTo(StreamOutput out) throws IOException
      Writes only the data of the block to a stream output. This method should be used when the type of the block is known during reading.
      Specified by:
      writeTo in interface Writeable
      Throws:
      IOException
    • writeTypedBlock

      static void writeTypedBlock(Block block, StreamOutput out) throws IOException
      Writes the type of the block followed by the block data to a stream output. This should be paired with readTypedBlock(BlockStreamInput)
      Throws:
      IOException
    • readTypedBlock

      static Block readTypedBlock(BlockStreamInput in) throws IOException
      Reads the block type and then the block data from a stream input This should be paired with writeTypedBlock(Block, StreamOutput)
      Throws:
      IOException
    • supportsAggregateMetricDoubleBlock

      static boolean supportsAggregateMetricDoubleBlock(TransportVersion version)