java.lang.Object
org.elasticsearch.simdvec.ESVectorUtil
-
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionstatic intandBitCount(byte[] a, byte[] b) AND bit count computed over signed bytes.static voidcalculateOSQGridPoints(float[] target, float[] interval, int points, float[] pts) Calculate the grid points for optimized-scalar quantizationstatic floatcalculateOSQLoss(float[] target, float[] interval, int points, float norm2, float lambda) Calculate the loss for optimized-scalar quantization for the given parameteresstatic voidcenterAndCalculateOSQStatsDp(float[] target, float[] centroid, float[] centered, float[] stats) Center the target vector and calculate the optimized-scalar quantization statisticsstatic voidcenterAndCalculateOSQStatsEuclidean(float[] target, float[] centroid, float[] centered, float[] stats) Center the target vector and calculate the optimized-scalar quantization statisticsstatic ES91Int4VectorsScorergetES91Int4VectorsScorer(org.apache.lucene.store.IndexInput input, int dimension) static ES91OSQVectorsScorergetES91OSQVectorsScorer(org.apache.lucene.store.IndexInput input, int dimension) static longipByteBinByte(byte[] q, byte[] d) static intipByteBit(byte[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a byte vector and the document vector is a bit vector.static floatipFloatBit(float[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a float vector and the document vector is a bit vector.static floatipFloatByte(float[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a float vector and the document vector is a byte vector.static floatsoarDistance(float[] v1, float[] centroid, float[] originalResidual, float soarLambda, float rnorm) calculates the soar distance for a vector and a centroidstatic voidsubtract(float[] v1, float[] v2, float[] result) Calculates the difference between two vectors and stores the result in a third vector.
-
Constructor Details
-
ESVectorUtil
public ESVectorUtil()
-
-
Method Details
-
getES91OSQVectorsScorer
public static ES91OSQVectorsScorer getES91OSQVectorsScorer(org.apache.lucene.store.IndexInput input, int dimension) throws IOException - Throws:
IOException
-
getES91Int4VectorsScorer
public static ES91Int4VectorsScorer getES91Int4VectorsScorer(org.apache.lucene.store.IndexInput input, int dimension) throws IOException - Throws:
IOException
-
ipByteBinByte
public static long ipByteBinByte(byte[] q, byte[] d) -
ipByteBit
public static int ipByteBit(byte[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a byte vector and the document vector is a bit vector. This will return the sum of the query vector values using the document vector as a mask. When comparing the bits with the bytes, they are done in "big endian" order. For example, if the byte vector is [1, 2, 3, 4, 5, 6, 7, 8] and the bit vector is [0b10000000], the inner product will be 1.0.- Parameters:
q- the query vectord- the document vector- Returns:
- the inner product of the two vectors
-
ipFloatBit
public static float ipFloatBit(float[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a float vector and the document vector is a bit vector. This will return the sum of the query vector values using the document vector as a mask. When comparing the bits with the floats, they are done in "big endian" order. For example, if the float vector is [1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0] and the bit vector is [0b10000000], the inner product will be 1.0.- Parameters:
q- the query vectord- the document vector- Returns:
- the inner product of the two vectors
-
ipFloatByte
public static float ipFloatByte(float[] q, byte[] d) Compute the inner product of two vectors, where the query vector is a float vector and the document vector is a byte vector.- Parameters:
q- the query vectord- the document vector- Returns:
- the inner product of the two vectors
-
andBitCount
public static int andBitCount(byte[] a, byte[] b) AND bit count computed over signed bytes. Copied from Lucene's XOR implementation- Parameters:
a- bytes containing a vectorb- bytes containing another vector, of the same dimension- Returns:
- the value of the AND bit count of the two vectors
-
calculateOSQLoss
public static float calculateOSQLoss(float[] target, float[] interval, int points, float norm2, float lambda) Calculate the loss for optimized-scalar quantization for the given parameteres- Parameters:
target- The vector being quantized, assumed to be centeredinterval- The interval for which to calculate the losspoints- the quantization pointsnorm2- The norm squared of the target vectorlambda- The lambda parameter for controlling anisotropic loss calculation- Returns:
- The loss for the given parameters
-
calculateOSQGridPoints
public static void calculateOSQGridPoints(float[] target, float[] interval, int points, float[] pts) Calculate the grid points for optimized-scalar quantization- Parameters:
target- The vector being quantized, assumed to be centeredinterval- The interval for which to calculate the grid pointspoints- the quantization pointspts- The array to store the grid points, must be of length 5
-
centerAndCalculateOSQStatsEuclidean
public static void centerAndCalculateOSQStatsEuclidean(float[] target, float[] centroid, float[] centered, float[] stats) Center the target vector and calculate the optimized-scalar quantization statistics- Parameters:
target- The vector being quantizedcentroid- The centroid of the target vectorcentered- The destination of the centered vector, will be overwrittenstats- The array to store the statistics, must be of length 5
-
centerAndCalculateOSQStatsDp
public static void centerAndCalculateOSQStatsDp(float[] target, float[] centroid, float[] centered, float[] stats) Center the target vector and calculate the optimized-scalar quantization statistics- Parameters:
target- The vector being quantizedcentroid- The centroid of the target vectorcentered- The destination of the centered vector, will be overwrittenstats- The array to store the statistics, must be of length 6
-
subtract
public static void subtract(float[] v1, float[] v2, float[] result) Calculates the difference between two vectors and stores the result in a third vector.- Parameters:
v1- the first vectorv2- the second vectorresult- the result vector, must be the same length as the input vectors
-
soarDistance
public static float soarDistance(float[] v1, float[] centroid, float[] originalResidual, float soarLambda, float rnorm) calculates the soar distance for a vector and a centroid- Parameters:
v1- the vectorcentroid- the centroidoriginalResidual- the residual with the actually nearest centroidsoarLambda- the lambda parameterrnorm- distance to the nearest centroid- Returns:
- the soar distance
-