org.apache.parquet.column.values.bitpacking

Class BytePacker

java.lang.Object

org.apache.parquet.column.values.bitpacking.BytePacker

public abstract class BytePacker
extends Object

Packs and unpacks into bytes

packing unpacking treats: - n values at a time (with n % 8 == 0) - bitWidth * (n/8) bytes at a time.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	unpackCount Number of integer values to be unpacked at a time.

Method Summary

Modifier and Type	Method and Description
int	getBitWidth()
int	getUnpackCount()
abstract void	pack32Values(int[] input, int inPos, byte[] output, int outPos) pack 32 values from input at inPos into bitWidth * 4 bytes in output at outPos.
abstract void	pack8Values(int[] input, int inPos, byte[] output, int outPos) pack 8 values from input at inPos into bitWidth bytes in output at outPos.
void	unpack32Values(byte[] input, int inPos, int[] output, int outPos) Deprecated.
abstract void	unpack32Values(ByteBuffer input, int inPos, int[] output, int outPos) unpack bitWidth * 4 bytes from input at inPos into 32 values in output at outPos.
void	unpack8Values(byte[] input, int inPos, int[] output, int outPos) Deprecated.
abstract void	unpack8Values(ByteBuffer input, int inPos, int[] output, int outPos) unpack bitWidth bytes from input at inPos into 8 values in output at outPos.
void	unpackValuesUsingVector(byte[] input, int inPos, int[] output, int outPos) unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.
void	unpackValuesUsingVector(ByteBuffer input, int inPos, int[] output, int outPos) unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

unpackCount

protected int unpackCount

Number of integer values to be unpacked at a time. unpackCount is a multiple of 8 For AVX512, the register is 512 bits, so output values at a time maybe different when different bitWidth.

Method Detail

getBitWidth

public final int getBitWidth()

Returns:

the width in bits used for encoding, also how many bytes are packed/unpacked at a time by pack8Values/unpack8Values

getUnpackCount

public int getUnpackCount()

pack8Values

public abstract void pack8Values(int[] input,
                                 int inPos,
                                 byte[] output,
                                 int outPos)

pack 8 values from input at inPos into bitWidth bytes in output at outPos. nextPosition: inPos += 8; outPos += getBitWidth()

Parameters:

input - the input values

inPos - where to read from in input

output - the output bytes

outPos - where to write to in output

pack32Values

public abstract void pack32Values(int[] input,
                                  int inPos,
                                  byte[] output,
                                  int outPos)

pack 32 values from input at inPos into bitWidth * 4 bytes in output at outPos. nextPosition: inPos += 32; outPos += getBitWidth() * 4

Parameters:

input - the input values

inPos - where to read from in input

output - the output bytes

outPos - where to write to in output

unpack8Values

public abstract void unpack8Values(ByteBuffer input,
                                   int inPos,
                                   int[] output,
                                   int outPos)

unpack bitWidth bytes from input at inPos into 8 values in output at outPos. nextPosition: inPos += getBitWidth(); outPos += 8

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output

unpack8Values

@Deprecated
public void unpack8Values(byte[] input,
                                      int inPos,
                                      int[] output,
                                      int outPos)

Deprecated.

Compatibility API

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output

unpack32Values

public abstract void unpack32Values(ByteBuffer input,
                                    int inPos,
                                    int[] output,
                                    int outPos)

unpack bitWidth * 4 bytes from input at inPos into 32 values in output at outPos. nextPosition: inPos += getBitWidth() * 4; outPos += 32

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output

unpack32Values

@Deprecated
public void unpack32Values(byte[] input,
                                       int inPos,
                                       int[] output,
                                       int outPos)

Deprecated.

Compatibility API

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output

unpackValuesUsingVector

public void unpackValuesUsingVector(byte[] input,
                                    int inPos,
                                    int[] output,
                                    int outPos)

unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output

unpackValuesUsingVector

public void unpackValuesUsingVector(ByteBuffer input,
                                    int inPos,
                                    int[] output,
                                    int outPos)

unpack bitWidth bytes from input at inPos into {unpackCount} values in output at outPos using Java Vector API.

Parameters:

input - the input bytes

inPos - where to read from in input

output - the output values

outPos - where to write to in output