Package org.apache.pinot.segment.local.io.writer.impl

Class FixedByteChunkSVForwardIndexWriter

  • All Implemented Interfaces:
    Closeable, AutoCloseable
    @NotThreadSafe
public class FixedByteChunkSVForwardIndexWriter
extends BaseChunkSVForwardIndexWriter
    Class to write out fixed length bytes into a single column. Client responsible to ensure that they call the correct set method that matches the sizeOfEntry. Avoiding checks here, as they can be expensive when called for each row. The layout of the file is as follows:

    Header Section:

    • Integer: File format version.
    • Integer: Total number of chunks.
    • Integer: Number of docs per chunk.
    • Integer: Length of entry (in bytes).
    • Integer: Total number of docs (version 2 onwards).
    • Integer: Compression type enum value (version 2 onwards).
    • Integer: Start offset of data header (version 2 onwards).
    • Integer array: Integer offsets for all chunks in the data (upto version 2), Long array: Long offsets for all chunks in the data (version 3 onwards)

    Individual Chunks:

    • Data bytes.
    Only sequential writes are supported.

    • Constructor Detail

      • FixedByteChunkSVForwardIndexWriter

        public FixedByteChunkSVForwardIndexWriter​(File file,
                                          ChunkCompressionType compressionType,
                                          int totalDocs,
                                          int numDocsPerChunk,
                                          int sizeOfEntry,
                                          int writerVersion)
                                   throws IOException
        Constructor for the class.
        Parameters:
        file - File to write to.
        compressionType - Type of compression to use.
        totalDocs - Total number of docs to write.
        numDocsPerChunk - Number of documents per chunk.
        sizeOfEntry - Size of entry (in bytes)
        writerVersion - writer format version
        Throws:
        FileNotFoundException - Throws FileNotFoundException if the specified file is not found.
        IOException - Throws IOException if there are any errors mapping the underlying ByteBuffer.

    • Method Detail

      • putInt

        public void putInt​(int value)

      • putLong

        public void putLong​(long value)

      • putFloat

        public void putFloat​(float value)

      • putDouble

        public void putDouble​(double value)

      • writeChunk

        protected void writeChunk()
        Description copied from class: BaseChunkSVForwardIndexWriter
        Helper method to compress and write the current chunk.
        • Chunk header is of fixed size, so fills out any remaining offsets for partially filled chunks.
        • Compresses (if required) and writes the chunk to the data file.
        • Updates the header with the current chunks offset.
        • Clears up the buffers, so that they can be reused.
        Overrides:
        writeChunk in class BaseChunkSVForwardIndexWriter