- All Superinterfaces:
org.apache.lucene.util.Accountable,AutoCloseable,BlockLoader.Block,Closeable,org.elasticsearch.core.RefCounted,org.elasticsearch.core.Releasable,Writeable
- All Known Subinterfaces:
AggregateMetricDoubleBlock,BooleanBlock,BytesRefBlock,DoubleBlock,FloatBlock,IntBlock,LongBlock
- All Known Implementing Classes:
AggregateMetricDoubleArrayBlock,BooleanArrayBlock,BooleanBigArrayBlock,BooleanVectorBlock,BytesRefArrayBlock,BytesRefVectorBlock,CompositeBlock,ConstantNullBlock,DocBlock,DoubleArrayBlock,DoubleBigArrayBlock,DoubleVectorBlock,FloatArrayBlock,FloatBigArrayBlock,FloatVectorBlock,IntArrayBlock,IntBigArrayBlock,IntVectorBlock,LongArrayBlock,LongBigArrayBlock,LongVectorBlock,OrdinalBytesRefBlock
Blocks can represent various shapes of underlying data. A Block can represent either sparse
or dense data. A Block can represent either single or multivalued data. A Block that represents
dense single-valued data can be viewed as a Vector.
Blocks are reference counted; to make a shallow copy of a block (e.g. if a Page contains
the same column twice), use RefCounted.incRef(). Before a block is garbage collected,
Releasable.close() must be called to release a block's resources; it must also be called one
additional time for each time RefCounted.incRef() was called. Calls to RefCounted.decRef() and
Releasable.close() are equivalent.
Block are immutable and can be passed between threads as long as no two threads hold a reference to the same block at the same time.
-
Nested Class Summary
Nested ClassesModifier and TypeInterfaceDescriptionstatic interfaceBuildsBlocks.static enumHow are multivalued fields ordered? Some operators can enable its optimization when mv_values are sorted ascending or de-duplicated.Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable
Writeable.Reader<V>, Writeable.Writer<V> -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final longThe maximum number of values that can be added to one position via lookup.static final intWe do not track memory for pages directly (only for single blocks), but the page memory overhead can still be significant, especially for pages containing thousands of blocks.static final bytestatic final bytestatic final bytestatic final byteSerialization type for blocks: 0 and 1 replace false/true used in pre-8.14static final byteFields inherited from interface org.apache.lucene.util.Accountable
NULL_ACCOUNTABLEFields inherited from interface org.elasticsearch.core.RefCounted
ALWAYS_REFERENCED -
Method Summary
Modifier and TypeMethodDescriptionvoidBefore passing a Block to another Driver, it is necessary to switch the owning block factory to its parent, which is associated with the global circuit breaker.booleanasVector()Returns an efficient dense single-value view of this block.The block factory associated with this block.booleanDoes this block have multivalued fields? UnlikemayHaveMultivaluedFields()this will never return a false positive.Returns the element type of this block.expand()Expand multivalued fields into one row per value.filter(int... positions) Creates a new block that only exposes the positions provided.intgetFirstValueIndex(int position) Gets the index of the first value for the given position.intReturns The number of positions in this block..intReturns The total number of values in this block not counting nulls..intgetValueCount(int position) Gets the number of values for the given position, possibly 0.default BlockinsertNulls(IntVector before) booleanisNull(int position) booleanTells if this block has been released.keepMask(BooleanVector mask) org.elasticsearch.core.ReleasableIterator<? extends Block> lookup(IntBlock positions, ByteSizeValue targetBlockSize) Builds an Iterator of newBlocks with the sameelementType()as this Block whose values are copied from positions in this Block.booleanCan this block have multivalued fields? Blocks that returnfalsewill never return more than one fromgetValueCount(int).booleandefault booleanAre multivalued fields de-duplicated in each positionHow are multivalued fields ordered?default booleanAre multivalued fields sorted ascending in each positionstatic BlockReads the block type and then the block data from a stream input This should be paired withwriteTypedBlock(Block, StreamOutput)static booleanvoidwriteTo(StreamOutput out) Writes only the data of the block to a stream output.static voidwriteTypedBlock(Block block, StreamOutput out) Writes the type of the block followed by the block data to a stream output.Methods inherited from interface org.apache.lucene.util.Accountable
getChildResources, ramBytesUsedMethods inherited from interface org.elasticsearch.core.RefCounted
decRef, hasReferences, incRef, mustIncRef, tryIncRefMethods inherited from interface org.elasticsearch.core.Releasable
close
-
Field Details
-
MAX_LOOKUP
static final long MAX_LOOKUPThe maximum number of values that can be added to one position via lookup. TODO maybe make this everywhere?- See Also:
-
PAGE_MEM_OVERHEAD_PER_BLOCK
static final int PAGE_MEM_OVERHEAD_PER_BLOCKWe do not track memory for pages directly (only for single blocks), but the page memory overhead can still be significant, especially for pages containing thousands of blocks. For now, we approximate this overhead, per block, using this value. The exact overhead per block would be (more correctly)RamUsageEstimator.NUM_BYTES_OBJECT_REF, but we approximate it withRamUsageEstimator.NUM_BYTES_OBJECT_ALIGNMENTto avoid further alignments to object size (at the end of the alignment, it would make no practical difference). -
SERIALIZE_BLOCK_VALUES
static final byte SERIALIZE_BLOCK_VALUESSerialization type for blocks: 0 and 1 replace false/true used in pre-8.14- See Also:
-
SERIALIZE_BLOCK_VECTOR
static final byte SERIALIZE_BLOCK_VECTOR- See Also:
-
SERIALIZE_BLOCK_ARRAY
static final byte SERIALIZE_BLOCK_ARRAY- See Also:
-
SERIALIZE_BLOCK_BIG_ARRAY
static final byte SERIALIZE_BLOCK_BIG_ARRAY- See Also:
-
SERIALIZE_BLOCK_ORDINAL
static final byte SERIALIZE_BLOCK_ORDINAL- See Also:
-
-
Method Details
-
asVector
Vector asVector()Returns an efficient dense single-value view of this block.. Null, if the block is not dense single-valued. That is, if mayHaveNulls returns true, or getTotalValueCount is not equal to getPositionCount.- Returns:
- an efficient dense single-value view of this block
-
getTotalValueCount
int getTotalValueCount()Returns The total number of values in this block not counting nulls..- Returns:
- The total number of values in this block not counting nulls.
-
getPositionCount
int getPositionCount()Returns The number of positions in this block..- Returns:
- The number of positions in this block.
-
getFirstValueIndex
int getFirstValueIndex(int position) Gets the index of the first value for the given position. -
getValueCount
int getValueCount(int position) Gets the number of values for the given position, possibly 0. -
elementType
ElementType elementType()Returns the element type of this block.- Returns:
- the element type of this block
-
blockFactory
BlockFactory blockFactory()The block factory associated with this block. -
allowPassingToDifferentDriver
void allowPassingToDifferentDriver()Before passing a Block to another Driver, it is necessary to switch the owning block factory to its parent, which is associated with the global circuit breaker. This ensures that when the new driver releases this Block, it returns memory directly to the parent block factory instead of the local block factory of this Block. This is important because the local block factory is not thread safe and doesn't support simultaneous access by more than one thread. -
isReleased
boolean isReleased()Tells if this block has been released. A block is released by calling itsReleasable.close()orRefCounted.decRef()methods.- Returns:
- true iff the block's reference count is zero.
-
isNull
boolean isNull(int position) - Parameters:
position- the position- Returns:
- true if the value stored at the given position is null, false otherwise
-
mayHaveNulls
boolean mayHaveNulls()- Returns:
- true if some values might be null. False, if all values are guaranteed to be not null.
-
areAllValuesNull
boolean areAllValuesNull()- Returns:
- true if all values in this block are guaranteed to be null.
-
mayHaveMultivaluedFields
boolean mayHaveMultivaluedFields()Can this block have multivalued fields? Blocks that returnfalsewill never return more than one fromgetValueCount(int). This may returntruefor Blocks that do not have multivalued fields, but it will always answer quickly. -
doesHaveMultivaluedFields
boolean doesHaveMultivaluedFields()Does this block have multivalued fields? UnlikemayHaveMultivaluedFields()this will never return a false positive. In other words, if this returnstruethen there are positions for whichgetValueCount(int)will return more than 1. This will answer quickly if it can but may have to check all positions. -
filter
Creates a new block that only exposes the positions provided.- Parameters:
positions- the positions to retain- Returns:
- a filtered block TODO: pass BlockFactory
-
keepMask
-
lookup
org.elasticsearch.core.ReleasableIterator<? extends Block> lookup(IntBlock positions, ByteSizeValue targetBlockSize) Builds an Iterator of newBlocks with the sameelementType()as this Block whose values are copied from positions in this Block. It has the same number ofpositionsas thepositionsparameter.For example, if this block contained
[a, b, [b, c]]and were called with the block[0, 1, 1, [1, 2]]then the result would be[a, b, b, [b, b, c]].This process produces
count(this) * count(positions)values per positions which could be quite large. Instead of returning a single Block, this returns an Iterator of Blocks containing all of the promised values.The returned
ReleasableIteratormay retain a reference to thepositionsparameter. Close it to release those references.This block is built using the same
BlockFactoryas was used to build thepositionsparameter. -
mvOrdering
Block.MvOrdering mvOrdering()How are multivalued fields ordered? -
mvDeduplicated
default boolean mvDeduplicated()Are multivalued fields de-duplicated in each position -
mvSortedAscending
default boolean mvSortedAscending()Are multivalued fields sorted ascending in each position -
expand
Block expand()Expand multivalued fields into one row per value. Returns the same block if there aren't any multivalued fields to expand. The returned block needs to be closed by the caller to release the block's resources. -
insertNulls
Build aBlockwith anullinsertedbeforeeach listed position.Note:
beforemust be non-decreasing. -
writeTo
Writes only the data of the block to a stream output. This method should be used when the type of the block is known during reading.- Specified by:
writeToin interfaceWriteable- Throws:
IOException
-
writeTypedBlock
Writes the type of the block followed by the block data to a stream output. This should be paired withreadTypedBlock(BlockStreamInput)- Throws:
IOException
-
readTypedBlock
Reads the block type and then the block data from a stream input This should be paired withwriteTypedBlock(Block, StreamOutput)- Throws:
IOException
-
supportsAggregateMetricDoubleBlock
-