Package org.apache.pinot.segment.local.segment.index.readers.forward

Class BaseChunkForwardIndexReader

java.lang.Object

org.apache.pinot.segment.local.segment.index.readers.forward.BaseChunkForwardIndexReader

All Implemented Interfaces:

Closeable, AutoCloseable, IndexReader, ForwardIndexReader<ChunkReaderContext>

Direct Known Subclasses:

FixedByteChunkMVForwardIndexReader, FixedByteChunkSVForwardIndexReader, FixedBytePower2ChunkSVForwardIndexReader, VarByteChunkMVForwardIndexReader, VarByteChunkSVForwardIndexReader

public abstract class BaseChunkForwardIndexReader
extends Object
implements ForwardIndexReader<ChunkReaderContext>

Base implementation for chunk-based raw (non-dictionary-encoded) forward index reader.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.pinot.segment.spi.index.reader.ForwardIndexReader

ForwardIndexReader.ByteRange

Field Summary

Fields

Modifier and Type	Field	Description
protected ChunkDecompressor	_chunkDecompressor
protected ChunkCompressionType	_compressionType
protected PinotDataBuffer	_dataBuffer
protected PinotDataBuffer	_dataHeader
protected int	_dataHeaderStart
protected int	_headerEntryChunkOffsetSize
protected boolean	_isCompressed
protected boolean	_isSingleValue
protected int	_lengthOfLongestEntry
protected int	_numChunks
protected int	_numDocsPerChunk
protected PinotDataBuffer	_rawData
protected int	_rawDataStart
protected FieldSpec.DataType	_storedType

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BaseChunkForwardIndexReader(PinotDataBuffer dataBuffer, FieldSpec.DataType storedType, boolean isSingleValue)

Method Summary

Modifier and Type	Method	Description
void	close()
protected ByteBuffer	decompressChunk(int chunkId, ChunkReaderContext context)
protected ByteBuffer	getChunkBuffer(int docId, ChunkReaderContext context)	Helper method to return the chunk buffer that contains the value at the given document id.
protected int	getChunkId(int docId)
protected long	getChunkPosition(int chunkId)	Helper method to get the offset of the chunk in the data.
protected long	getChunkPositionAndRecordRanges(int chunkId, List<ForwardIndexReader.ByteRange> ranges)
ChunkCompressionType	getCompressionType()
int	getLengthOfLongestEntry()
FieldSpec.DataType	getStoredType()
protected long	getValueEndOffsetAndRecordRanges(int chunkId, int chunkRowId, int rowOffsetSize, long chunkStartOffset, List<ForwardIndexReader.ByteRange> ranges)
boolean	isDictionaryEncoded()
boolean	isSingleValue()
void	readValuesSV(int[] docIds, int length, double[] values, ChunkReaderContext context)
void	readValuesSV(int[] docIds, int length, float[] values, ChunkReaderContext context)
void	readValuesSV(int[] docIds, int length, int[] values, ChunkReaderContext context)
void	readValuesSV(int[] docIds, int length, long[] values, ChunkReaderContext context)
protected void	recordChunkRanges(int chunkId, ChunkReaderContext context, List<ForwardIndexReader.ByteRange> ranges)
protected void	recordDocIdRanges(int docId, ChunkReaderContext context, List<ForwardIndexReader.ByteRange> ranges)
protected void	recordDocIdRangesUncompressed(int docId, int rowOffsetSize, List<ForwardIndexReader.ByteRange> ranges)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pinot.segment.spi.index.reader.ForwardIndexReader

createContext, getBigDecimal, getBytes, getBytesMV, getBytesMV, getDictId, getDictIdCompressionType, getDictIdMV, getDictIdMV, getDocLength, getDouble, getDoubleMV, getDoubleMV, getFloat, getFloatMV, getFloatMV, getInt, getIntMV, getIntMV, getLong, getLongMV, getLongMV, getNumValuesMV, getRawDataStartOffset, getString, getStringMV, getStringMV, isBufferByteRangeInfoSupported, isDocLengthInBits, isFixedOffsetMappingType, readDictIds, readValuesMV, readValuesMV, readValuesMV, readValuesMV, readValuesMV, readValuesMV, readValuesSV, recordDocIdByteRanges

Field Detail

_dataBuffer

protected final PinotDataBuffer _dataBuffer

_storedType

protected final FieldSpec.DataType _storedType

_numChunks

protected final int _numChunks

_numDocsPerChunk

protected final int _numDocsPerChunk

_lengthOfLongestEntry

protected final int _lengthOfLongestEntry

_isCompressed

protected final boolean _isCompressed

_compressionType

protected final ChunkCompressionType _compressionType

_chunkDecompressor

protected final ChunkDecompressor _chunkDecompressor

_dataHeader

protected final PinotDataBuffer _dataHeader

_headerEntryChunkOffsetSize

protected final int _headerEntryChunkOffsetSize

_rawData

protected final PinotDataBuffer _rawData

_isSingleValue

protected final boolean _isSingleValue

_dataHeaderStart

protected final int _dataHeaderStart

_rawDataStart

protected final int _rawDataStart

Constructor Detail

BaseChunkForwardIndexReader

protected BaseChunkForwardIndexReader(PinotDataBuffer dataBuffer,
                                      FieldSpec.DataType storedType,
                                      boolean isSingleValue)

Method Detail

getChunkBuffer

protected ByteBuffer getChunkBuffer(int docId,
                                    ChunkReaderContext context)

Helper method to return the chunk buffer that contains the value at the given document id.

• If the chunk already exists in the reader context, returns the same.
• Otherwise, loads the chunk for the row, and sets it in the reader context.

Parameters:

docId - Document id

context - Reader context

Returns:

Chunk for the row

getChunkId

protected int getChunkId(int docId)

recordDocIdRangesUncompressed

protected void recordDocIdRangesUncompressed(int docId,
                                             int rowOffsetSize,
                                             List<ForwardIndexReader.ByteRange> ranges)

getValueEndOffsetAndRecordRanges

protected long getValueEndOffsetAndRecordRanges(int chunkId,
                                                int chunkRowId,
                                                int rowOffsetSize,
                                                long chunkStartOffset,
                                                List<ForwardIndexReader.ByteRange> ranges)

recordDocIdRanges

protected void recordDocIdRanges(int docId,
                                 ChunkReaderContext context,
                                 List<ForwardIndexReader.ByteRange> ranges)

recordChunkRanges

protected void recordChunkRanges(int chunkId,
                                 ChunkReaderContext context,
                                 List<ForwardIndexReader.ByteRange> ranges)

decompressChunk

protected ByteBuffer decompressChunk(int chunkId,
                                     ChunkReaderContext context)

getChunkPosition

protected long getChunkPosition(int chunkId)

Helper method to get the offset of the chunk in the data.

Parameters:

chunkId - Id of the chunk for which to return the position.

Returns:

Position (offset) of the chunk in the data.

getChunkPositionAndRecordRanges

protected long getChunkPositionAndRecordRanges(int chunkId,
                                               List<ForwardIndexReader.ByteRange> ranges)

isDictionaryEncoded

public boolean isDictionaryEncoded()

Specified by:

isDictionaryEncoded in interface ForwardIndexReader<ChunkReaderContext>

isSingleValue

public boolean isSingleValue()

Specified by:

isSingleValue in interface ForwardIndexReader<ChunkReaderContext>

getStoredType

public FieldSpec.DataType getStoredType()

Specified by:

getStoredType in interface ForwardIndexReader<ChunkReaderContext>

getCompressionType

public ChunkCompressionType getCompressionType()

Specified by:

getCompressionType in interface ForwardIndexReader<ChunkReaderContext>

getLengthOfLongestEntry

public int getLengthOfLongestEntry()

Specified by:

getLengthOfLongestEntry in interface ForwardIndexReader<ChunkReaderContext>

readValuesSV

public void readValuesSV(int[] docIds,
                         int length,
                         int[] values,
                         ChunkReaderContext context)

Specified by:

readValuesSV in interface ForwardIndexReader<ChunkReaderContext>

readValuesSV

public void readValuesSV(int[] docIds,
                         int length,
                         long[] values,
                         ChunkReaderContext context)

Specified by:

readValuesSV in interface ForwardIndexReader<ChunkReaderContext>

readValuesSV

public void readValuesSV(int[] docIds,
                         int length,
                         float[] values,
                         ChunkReaderContext context)

Specified by:

readValuesSV in interface ForwardIndexReader<ChunkReaderContext>

readValuesSV

public void readValuesSV(int[] docIds,
                         int length,
                         double[] values,
                         ChunkReaderContext context)

Specified by:

readValuesSV in interface ForwardIndexReader<ChunkReaderContext>

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable