org.apache.hudi.io.hfile

Class HFileBlock

java.lang.Object

org.apache.hudi.io.hfile.HFileBlock

Direct Known Subclasses:

HFileDataBlock, HFileFileInfoBlock, HFileIndexBlock, HFileLeafIndexBlock, HFileMetaBlock

public abstract class HFileBlock
extends Object

Represents a block in a HFile. The types of blocks are defined in HFileBlockType.

Field Summary

Fields

Modifier and Type	Field and Description
protected byte[]	byteBuff
protected int	bytesPerChecksum
protected byte[]	compressedByteBuff
protected HFileContext	context
static int	HFILEBLOCK_HEADER_SIZE
static int	HFILEBLOCK_HEADER_SIZE_NO_CHECKSUM
protected int	onDiskSizeWithoutHeader
protected int	sizeCheckSum
protected int	startOffsetInBuff
protected int	startOffsetInCompressedBuff
protected int	uncompressedEndOffset
protected int	uncompressedSizeWithoutHeader

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HFileBlock(HFileContext context, HFileBlockType blockType, byte[] byteBuff, int startOffsetInBuff) Initialize HFileBlock for read.
protected	HFileBlock(HFileContext context, HFileBlockType blockType, long previousBlockOffsetForWrite) Initialize HFileBlock for write.

Method Summary

Modifier and Type	Method and Description
protected byte[]	allocateBufferForUnpacking() Allocates new byte buffer for the uncompressed bytes.
HFileBlockType	getBlockType()
byte[]	getByteBuff()
int	getOnDiskSizeWithHeader()
protected long	getStartOffsetInBuffForWrite() Gets start offset of the block in the buffer.
protected abstract ByteBuffer	getUncompressedBlockDataToWrite() Returns serialized "data" part of the block.
static HFileBlock	parse(HFileContext context, byte[] byteBuff, int startOffsetInBuff) Parses the HFile block header and returns the HFileBlock instance based on the input.
ByteBuffer	serialize() Return serialized block including header, data, checksum.
protected void	setStartOffsetInBuffForWrite(long startOffsetInBuffForWrite) Sets start offset of the block in the buffer.
void	unpack() Decodes and decompresses the block content if the block content is compressed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HFILEBLOCK_HEADER_SIZE_NO_CHECKSUM

public static final int HFILEBLOCK_HEADER_SIZE_NO_CHECKSUM

See Also:

Constant Field Values

HFILEBLOCK_HEADER_SIZE

public static final int HFILEBLOCK_HEADER_SIZE

See Also:

Constant Field Values

context

protected final HFileContext context

byteBuff

protected final byte[] byteBuff

startOffsetInBuff

protected final int startOffsetInBuff

sizeCheckSum

protected final int sizeCheckSum

uncompressedEndOffset

protected final int uncompressedEndOffset

onDiskSizeWithoutHeader

protected final int onDiskSizeWithoutHeader

uncompressedSizeWithoutHeader

protected final int uncompressedSizeWithoutHeader

bytesPerChecksum

protected final int bytesPerChecksum

compressedByteBuff

protected byte[] compressedByteBuff

startOffsetInCompressedBuff

protected int startOffsetInCompressedBuff

Constructor Detail

HFileBlock

protected HFileBlock(HFileContext context,
                     HFileBlockType blockType,
                     byte[] byteBuff,
                     int startOffsetInBuff)

Initialize HFileBlock for read.

HFileBlock

protected HFileBlock(HFileContext context,
                     HFileBlockType blockType,
                     long previousBlockOffsetForWrite)

Initialize HFileBlock for write.

Method Detail

parse

public static HFileBlock parse(HFileContext context,
                               byte[] byteBuff,
                               int startOffsetInBuff)
                        throws IOException

Parses the HFile block header and returns the HFileBlock instance based on the input.

Parameters:

context - HFile context.

byteBuff - input data.

startOffsetInBuff - offset to start parsing.

Returns:

the HFileBlock instance based on the input.

Throws:

IOException - if the block cannot be parsed.

getBlockType

public HFileBlockType getBlockType()

getByteBuff

public byte[] getByteBuff()

getOnDiskSizeWithHeader

public int getOnDiskSizeWithHeader()

unpack

public void unpack()
            throws IOException

Decodes and decompresses the block content if the block content is compressed.

This must be called for an encoded and compressed block before any reads.

Throws:

IOException - upon decoding and decompression error.

allocateBufferForUnpacking

protected byte[] allocateBufferForUnpacking()

Allocates new byte buffer for the uncompressed bytes.

Returns:

a new byte array based on the size of uncompressed data, holding the same header bytes.

getUncompressedBlockDataToWrite

protected abstract ByteBuffer getUncompressedBlockDataToWrite()
                                                       throws IOException

Returns serialized "data" part of the block. This function must be implemented by each block type separately.

Throws:

IOException

serialize

public ByteBuffer serialize()
                     throws IOException

Return serialized block including header, data, checksum.

Throws:

IOException

setStartOffsetInBuffForWrite

protected void setStartOffsetInBuffForWrite(long startOffsetInBuffForWrite)

Sets start offset of the block in the buffer.

getStartOffsetInBuffForWrite

protected long getStartOffsetInBuffForWrite()

Gets start offset of the block in the buffer.