Package org.apache.pinot.segment.local.io.writer.impl

Class BaseChunkSVForwardIndexWriter

java.lang.Object

org.apache.pinot.segment.local.io.writer.impl.BaseChunkSVForwardIndexWriter

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

FixedByteChunkSVForwardIndexWriter, VarByteChunkSVForwardIndexWriter

public abstract class BaseChunkSVForwardIndexWriter
extends Object
implements Closeable

Base implementation for chunk-based single-value raw (non-dictionary-encoded) forward index writer.

Field Summary

Fields

Modifier and Type	Field	Description
protected ByteBuffer	_chunkBuffer
protected ChunkCompressor	_chunkCompressor
protected int	_chunkSize
protected ByteBuffer	_compressedBuffer
protected FileChannel	_dataFile
protected long	_dataOffset
protected ByteBuffer	_header
static int	CURRENT_VERSION
static int	DEFAULT_VERSION

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	BaseChunkSVForwardIndexWriter(File file, ChunkCompressionType compressionType, int totalDocs, int numDocsPerChunk, long chunkSize, int sizeOfEntry, int version, boolean fixed)	Constructor for the class.

Method Summary

Modifier and Type	Method	Description
void	close()
static int	getHeaderEntryChunkOffsetSize(int version)
protected void	writeChunk()	Helper method to compress and write the current chunk.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_VERSION

public static final int DEFAULT_VERSION

See Also:

Constant Field Values

CURRENT_VERSION

public static final int CURRENT_VERSION

See Also:

Constant Field Values

_dataFile

protected final FileChannel _dataFile

_header

protected ByteBuffer _header

_chunkBuffer

protected final ByteBuffer _chunkBuffer

_compressedBuffer

protected final ByteBuffer _compressedBuffer

_chunkCompressor

protected final ChunkCompressor _chunkCompressor

_chunkSize

protected int _chunkSize

_dataOffset

protected long _dataOffset

Constructor Detail

BaseChunkSVForwardIndexWriter

protected BaseChunkSVForwardIndexWriter(File file,
                                        ChunkCompressionType compressionType,
                                        int totalDocs,
                                        int numDocsPerChunk,
                                        long chunkSize,
                                        int sizeOfEntry,
                                        int version,
                                        boolean fixed)
                                 throws IOException

Constructor for the class.

Parameters:

file - Data file to write into

compressionType - Type of compression

totalDocs - Total docs to write

numDocsPerChunk - Number of docs per data chunk

chunkSize - Size of chunk

sizeOfEntry - Size of entry (in bytes), max size for variable byte implementation.

version - version of File

fixed - if the data type is fixed width (required for version validation)

Throws:

IOException - if the file isn't found or can't be mapped

Method Detail

getHeaderEntryChunkOffsetSize

public static int getHeaderEntryChunkOffsetSize(int version)

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException

writeChunk

protected void writeChunk()

Helper method to compress and write the current chunk.

• Chunk header is of fixed size, so fills out any remaining offsets for partially filled chunks.
• Compresses (if required) and writes the chunk to the data file.
• Updates the header with the current chunks offset.
• Clears up the buffers, so that they can be reused.