Package org.apache.druid.segment.data

Class FrontCodedIndexedWriter

java.lang.Object

org.apache.druid.segment.data.FrontCodedIndexedWriter

All Implemented Interfaces:

DictionaryWriter<byte[]>, Serializer

public class FrontCodedIndexedWriter
extends Object
implements DictionaryWriter<byte[]>

DictionaryWriter for a FrontCodedIndexed, written to a SegmentWriteOutMedium. Values MUST be added to this dictionary writer in sorted order, which is enforced. Front coding is a type of delta encoding for byte arrays, where values are grouped into buckets. The first value of the bucket is written entirely, and remaining values are stored as pairs of an integer which indicates how much of the first byte array of the bucket to use as a prefix, (or the preceding value of the bucket if using 'incremental' buckets) followed by the remaining value bytes after the prefix. This writer is designed for use with UTF-8 encoded strings that are written in an order compatible with String.compareTo(String).

See Also:

for additional details.

Constructor Summary

Constructors

Constructor	Description
FrontCodedIndexedWriter(SegmentWriteOutMedium segmentWriteOutMedium, ByteOrder byteOrder, int bucketSize, byte version)

Method Summary

Modifier and Type	Method	Description
byte[]	get(int index)
int	getCardinality()
long	getSerializedSize()	Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher) call.
boolean	isSorted()
void	open()
void	write(byte[] value)
static int	writeBucketV0(ByteBuffer buffer, byte[][] values, int numValues)	Write bucket of values to a ByteBuffer.
static int	writeBucketV1(ByteBuffer buffer, byte[][] values, int numValues)	Write bucket of values to a ByteBuffer.
void	writeTo(WritableByteChannel channel, FileSmoosher smoosher)	Writes serialized form of this object to the given channel.
static int	writeValue(ByteBuffer buffer, byte[] bytes)	Write a variable length byte[] value to a ByteBuffer, storing the length as a VByte encoded integer followed by the value itself.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FrontCodedIndexedWriter

public FrontCodedIndexedWriter(SegmentWriteOutMedium segmentWriteOutMedium,
                               ByteOrder byteOrder,
                               int bucketSize,
                               byte version)

Method Detail

open

public void open()
          throws IOException

Specified by:

open in interface DictionaryWriter<byte[]>

Throws:

IOException

write

public void write(@Nullable
                  byte[] value)
           throws IOException

Specified by:

write in interface DictionaryWriter<byte[]>

Throws:

IOException

getSerializedSize

public long getSerializedSize()
                       throws IOException

Description copied from interface: Serializer

Returns the number of bytes, that this Serializer will write to the output _channel_ (not smoosher) on a Serializer.writeTo(java.nio.channels.WritableByteChannel, org.apache.druid.java.util.common.io.smoosh.FileSmoosher) call.

Specified by:

getSerializedSize in interface Serializer

Throws:

IOException

writeTo

public void writeTo(WritableByteChannel channel,
                    FileSmoosher smoosher)
             throws IOException

Description copied from interface: Serializer

Writes serialized form of this object to the given channel. If parallel data streams are needed, they could be created with the provided smoosher.

Specified by:

writeTo in interface Serializer

Throws:

IOException

isSorted

public boolean isSorted()

Specified by:

isSorted in interface DictionaryWriter<byte[]>

get

@Nullable
public byte[] get(int index)
           throws IOException

Specified by:

get in interface DictionaryWriter<byte[]>

Throws:

IOException

getCardinality

public int getCardinality()

Specified by:

getCardinality in interface DictionaryWriter<byte[]>

writeBucketV0

public static int writeBucketV0(ByteBuffer buffer,
                                byte[][] values,
                                int numValues)

Write bucket of values to a ByteBuffer. The first value is written completely, subsequent values are written with an integer to indicate how much of the first value in the bucket is a prefix of the value, followed by the remaining bytes of the value. Uses VByte encoded integers to indicate prefix length and value length.

writeBucketV1

public static int writeBucketV1(ByteBuffer buffer,
                                byte[][] values,
                                int numValues)

Write bucket of values to a ByteBuffer. The first value is written completely, subsequent values are written with an integer to indicate how much of the preceding value in the bucket is a prefix of the value, followed by the remaining bytes of the value. Uses VByte encoded integers to indicate prefix length and value length.

writeValue

public static int writeValue(ByteBuffer buffer,
                             byte[] bytes)

Write a variable length byte[] value to a ByteBuffer, storing the length as a VByte encoded integer followed by the value itself. Returns the number of bytes written to the buffer. This method returns a negative value if there is no room available in the buffer, so that it can be grown if needed.