org.apache.parquet.hadoop.codec

Class NonBlockedCompressor

java.lang.Object

org.apache.parquet.hadoop.codec.NonBlockedCompressor

All Implemented Interfaces:

org.apache.hadoop.io.compress.Compressor

Direct Known Subclasses:

Lz4RawCompressor, SnappyCompressor

public abstract class NonBlockedCompressor
extends Object
implements org.apache.hadoop.io.compress.Compressor

This class is a wrapper around the underlying compressor. It always consumes the entire input in setInput and compresses it as one compressed block.

Constructor Summary

Constructors

Constructor and Description
NonBlockedCompressor()

Method Summary

Modifier and Type	Method and Description
int	compress(byte[] buffer, int off, int len) Fills specified buffer with compressed data.
protected abstract int	compress(ByteBuffer uncompressed, ByteBuffer compressed) Compress the content in the given input buffer.
void	end()
void	finish()
boolean	finished()
long	getBytesRead()
long	getBytesWritten()
protected abstract int	maxCompressedLength(int byteSize) Get the maximum byte size needed for compressing data of the given byte size.
boolean	needsInput()
void	reinit(org.apache.hadoop.conf.Configuration c)
void	reset()
void	setDictionary(byte[] dictionary, int off, int len)
void	setInput(byte[] buffer, int off, int len)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

NonBlockedCompressor

public NonBlockedCompressor()

Method Detail

compress

public int compress(byte[] buffer,
                    int off,
                    int len)
             throws IOException

Fills specified buffer with compressed data. Returns actual number of bytes of compressed data. A return value of 0 indicates that needsInput() should be called in order to determine if more input data is required.

Specified by:

compress in interface org.apache.hadoop.io.compress.Compressor

Parameters:

buffer - Buffer for the compressed data

off - Start offset of the data

len - Size of the buffer

Returns:

The actual number of bytes of compressed data.

Throws:

IOException

setInput

public void setInput(byte[] buffer,
                     int off,
                     int len)

Specified by:

setInput in interface org.apache.hadoop.io.compress.Compressor

end

public void end()

Specified by:

end in interface org.apache.hadoop.io.compress.Compressor

finish

public void finish()

Specified by:

finish in interface org.apache.hadoop.io.compress.Compressor

finished

public boolean finished()

Specified by:

finished in interface org.apache.hadoop.io.compress.Compressor

getBytesRead

public long getBytesRead()

Specified by:

getBytesRead in interface org.apache.hadoop.io.compress.Compressor

getBytesWritten

public long getBytesWritten()

Specified by:

getBytesWritten in interface org.apache.hadoop.io.compress.Compressor

needsInput

public boolean needsInput()

Specified by:

needsInput in interface org.apache.hadoop.io.compress.Compressor

reinit

public void reinit(org.apache.hadoop.conf.Configuration c)

Specified by:

reinit in interface org.apache.hadoop.io.compress.Compressor

reset

public void reset()

Specified by:

reset in interface org.apache.hadoop.io.compress.Compressor

setDictionary

public void setDictionary(byte[] dictionary,
                          int off,
                          int len)

Specified by:

setDictionary in interface org.apache.hadoop.io.compress.Compressor

maxCompressedLength

protected abstract int maxCompressedLength(int byteSize)

Get the maximum byte size needed for compressing data of the given byte size.

Parameters:

byteSize - byte size of the data to compress

Returns:

maximum byte size of the compressed data

compress

protected abstract int compress(ByteBuffer uncompressed,
                                ByteBuffer compressed)
                         throws IOException

Compress the content in the given input buffer. After the compression, you can retrieve the compressed data from the output buffer [pos() ... limit()) (compressed data size = limit() - pos() = remaining())

Parameters:

uncompressed - buffer[pos() ... limit()) containing the input data

compressed - output of the compressed data. Uses range [pos()..].

Returns:

byte size of the compressed data.

Throws:

IOException