Package org.apache.pinot.segment.local.segment.creator.impl.vector.lucene95

Class HnswVectorsFormat

java.lang.Object

org.apache.lucene.codecs.KnnVectorsFormat

org.apache.pinot.segment.local.segment.creator.impl.vector.lucene95.HnswVectorsFormat

All Implemented Interfaces:

org.apache.lucene.util.NamedSPILoader.NamedSPI

public final class HnswVectorsFormat
extends org.apache.lucene.codecs.KnnVectorsFormat

Extend Lucene 9.5 vector format to support HNSW graph The major change here is to allow higher dimension vectors

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_MAX_DIMENSIONS	The maximum number of dimensions supported by this format.

Fields inherited from class org.apache.lucene.codecs.KnnVectorsFormat

EMPTY

Constructor Summary

Constructors

Constructor	Description
HnswVectorsFormat(int maxConn, int beamWidth, int maxDimensions)	Constructs a format using the given graph construction parameters.

Method Summary

Modifier and Type	Method	Description
org.apache.lucene.codecs.KnnVectorsReader	fieldsReader(org.apache.lucene.index.SegmentReadState state)
org.apache.lucene.codecs.KnnVectorsWriter	fieldsWriter(org.apache.lucene.index.SegmentWriteState state)
int	getMaxDimensions(String fieldName)
String	toString()

Methods inherited from class org.apache.lucene.codecs.KnnVectorsFormat

forName, getName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_MAX_DIMENSIONS

public static final int DEFAULT_MAX_DIMENSIONS

The maximum number of dimensions supported by this format. This is a limitation of the underlying implementation of HnswGraph which uses a fixed size array to store the vector values.

See Also:

Constant Field Values

Constructor Detail

HnswVectorsFormat

public HnswVectorsFormat(int maxConn,
                         int beamWidth,
                         int maxDimensions)

Constructs a format using the given graph construction parameters.

Parameters:

maxConn - the maximum number of connections to a node in the HNSW graph

beamWidth - the size of the queue maintained during graph construction.

maxDimensions - the maximum number of dimensions supported by this format

Method Detail

fieldsWriter

public org.apache.lucene.codecs.KnnVectorsWriter fieldsWriter(org.apache.lucene.index.SegmentWriteState state)
                                                       throws IOException

Specified by:

fieldsWriter in class org.apache.lucene.codecs.KnnVectorsFormat

Throws:

IOException

fieldsReader

public org.apache.lucene.codecs.KnnVectorsReader fieldsReader(org.apache.lucene.index.SegmentReadState state)
                                                       throws IOException

Specified by:

fieldsReader in class org.apache.lucene.codecs.KnnVectorsFormat

Throws:

IOException

getMaxDimensions

public int getMaxDimensions(String fieldName)

Overrides:

getMaxDimensions in class org.apache.lucene.codecs.KnnVectorsFormat

toString

public String toString()

Overrides:

toString in class Object