Module org.elasticsearch.compute
Package org.elasticsearch.compute.lucene
Enum Class DataPartitioning
- All Implemented Interfaces:
Serializable,Comparable<DataPartitioning>,Constable
How we partition the data across
Drivers. Each request forks into
min(1.5 * cpus, partition_count) threads on the data node. More partitions
allow us to bring more threads to bear on CPU intensive data node side tasks.-
Nested Class Summary
Nested classes/interfaces inherited from class java.lang.Enum
Enum.EnumDesc<E extends Enum<E>> -
Enum Constant Summary
Enum ConstantsEnum ConstantDescriptionAutomatically select the data partitioning based on the query and index.Partitions into dynamic-sized slices to improve CPU utilization while keeping overhead low.Partition on segment boundaries, this doesn't allow forking to as many CPUs asDOCbut it has much lower overhead.Make one partition per shard. -
Method Summary
Modifier and TypeMethodDescriptionstatic DataPartitioningReturns the enum constant of this class with the specified name.static DataPartitioning[]values()Returns an array containing the constants of this enum class, in the order they are declared.
-
Enum Constant Details
-
AUTO
-
SHARD
Make one partition per shard. This is generally the slowest option, but it has the lowest CPU overhead. -
SEGMENT
Partition on segment boundaries, this doesn't allow forking to as many CPUs asDOCbut it has much lower overhead.It packs segments smaller than
LuceneSliceQueue.MAX_DOCS_PER_SLICEdocs together into a partition. Larger segments get their own partition. Each slice contains no more thanLuceneSliceQueue.MAX_SEGMENTS_PER_SLICE. -
DOC
Partitions into dynamic-sized slices to improve CPU utilization while keeping overhead low. This approach is more flexible thanSEGMENTand works as follows:- The slice size starts from a desired size based on
task_concurrencybut is capped at aroundLuceneSliceQueue.MAX_DOCS_PER_SLICE. This prevents poor CPU usage when matching documents are clustered together. - For small and medium segments (less than five times the desired slice size), it uses a
slightly different
SEGMENTstrategy, which also splits segments that are larger than the desired size. SeeIndexSearcher.slices(List, int, int, boolean). - For very large segments, multiple segments are not combined into a single slice. This allows
one driver to process an entire large segment until other drivers steal the work after finishing
their own tasks. See
LuceneSliceQueue.nextSlice(LuceneSlice).
- The slice size starts from a desired size based on
-
-
Method Details
-
values
Returns an array containing the constants of this enum class, in the order they are declared.- Returns:
- an array containing the constants of this enum class, in the order they are declared
-
valueOf
Returns the enum constant of this class with the specified name. The string must match exactly an identifier used to declare an enum constant in this class. (Extraneous whitespace characters are not permitted.)- Parameters:
name- the name of the enum constant to be returned.- Returns:
- the enum constant with the specified name
- Throws:
IllegalArgumentException- if this enum class has no constant with the specified nameNullPointerException- if the argument is null
-