Package org.apache.pinot.segment.local.io.writer.impl

Class VarByteChunkForwardIndexWriterV4

java.lang.Object

org.apache.pinot.segment.local.io.writer.impl.VarByteChunkForwardIndexWriterV4

All Implemented Interfaces:

Closeable, AutoCloseable, VarByteChunkWriter

@NotThreadSafe
public class VarByteChunkForwardIndexWriterV4
extends Object
implements VarByteChunkWriter

Chunk-based raw (non-dictionary-encoded) forward index writer where each chunk contains variable number of docs, and the entries are variable length.

The layout of the file is as follows:

• Header Section
  • File format version (int)
  • Target decompressed chunk size (int)
  • Compression type enum value (int)
  • Start offset of chunk data (int)
  • Data header (for each chunk)
    • First docId in the chunk (int), where MSB is used to mark huge chunk
    • Start offset of the chunk (unsigned int)
• Individual Chunks
  • Regular chunk
    • Header Section: start offsets (stored as int) of the entry within the data section
    • Data Section
  • Huge chunk: contains one single value

Field Summary

Fields

Modifier and Type	Field	Description
static int	VERSION

Constructor Summary

Constructors

Constructor	Description
VarByteChunkForwardIndexWriterV4(File file, ChunkCompressionType compressionType, int chunkSize)

Method Summary

Modifier and Type	Method	Description
void	close()
void	putBigDecimal(BigDecimal bigDecimal)
void	putBytes(byte[] bytes)
void	putBytesMV(byte[][] values)
void	putString(String string)
void	putStringMV(String[] values)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

VERSION

public static final int VERSION

See Also:

Constant Field Values

Constructor Detail

VarByteChunkForwardIndexWriterV4

public VarByteChunkForwardIndexWriterV4(File file,
                                        ChunkCompressionType compressionType,
                                        int chunkSize)
                                 throws IOException

Throws:

IOException

Method Detail

putBigDecimal

public void putBigDecimal(BigDecimal bigDecimal)

Specified by:

putBigDecimal in interface VarByteChunkWriter

putString

public void putString(String string)

Specified by:

putString in interface VarByteChunkWriter

putBytes

public void putBytes(byte[] bytes)

Specified by:

putBytes in interface VarByteChunkWriter

putStringMV

public void putStringMV(String[] values)

Specified by:

putStringMV in interface VarByteChunkWriter

putBytesMV

public void putBytesMV(byte[][] values)

Specified by:

putBytesMV in interface VarByteChunkWriter

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException