ai.rapids.cudf.nvcomp

Class BatchedLZ4Compressor

java.lang.Object

ai.rapids.cudf.nvcomp.BatchedLZ4Compressor

public class BatchedLZ4Compressor
extends Object

Multi-buffer LZ4 compressor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BatchedLZ4Compressor.BatchedCompressionResult Describes a batched compression result

Constructor Summary

Constructors

Constructor and Description
BatchedLZ4Compressor()

Method Summary

Modifier and Type	Method and Description
static BatchedLZ4Compressor.BatchedCompressionResult	compress(BaseDeviceMemoryBuffer[] inputs, long chunkSize, Cuda.Stream stream) Compress a batch of buffers with LZ4
static void	compressAsync(HostMemoryBuffer compressedSizesOutputBuffer, BaseDeviceMemoryBuffer[] inputs, long chunkSize, BaseDeviceMemoryBuffer tempBuffer, BaseDeviceMemoryBuffer[] outputs, Cuda.Stream stream) Asynchronously compress a batch of input buffers.
static long	getCompressedSizesBufferSize(int numBuffers) Calculates the minimum size in bytes necessary to store the compressed output sizes when performing an asynchronous batch compression.
static long[]	getOutputSizes(BaseDeviceMemoryBuffer[] inputs, long chunkSize, BaseDeviceMemoryBuffer tempBuffer) Get the amount of output storage space required to compress a batch of buffers.
static long	getTempSize(BaseDeviceMemoryBuffer[] inputs, long chunkSize) Get the amount of temporary storage space required to compress a batch of buffers.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BatchedLZ4Compressor

public BatchedLZ4Compressor()

Method Detail

getTempSize

public static long getTempSize(BaseDeviceMemoryBuffer[] inputs,
                               long chunkSize)

Get the amount of temporary storage space required to compress a batch of buffers.

Parameters:

inputs - batch of data buffers to be individually compressed

chunkSize - compression chunk size to use

Returns:

amount in bytes of temporary storage space required to compress the batch

getOutputSizes

public static long[] getOutputSizes(BaseDeviceMemoryBuffer[] inputs,
                                    long chunkSize,
                                    BaseDeviceMemoryBuffer tempBuffer)

Get the amount of output storage space required to compress a batch of buffers.

Parameters:

inputs - batch of data buffers to be individually compressed

chunkSize - compression chunk size to use

tempBuffer - temporary storage space

Returns:

amount in bytes of output storage space corresponding to each input buffer in the batch

getCompressedSizesBufferSize

public static long getCompressedSizesBufferSize(int numBuffers)

Calculates the minimum size in bytes necessary to store the compressed output sizes when performing an asynchronous batch compression.

Parameters:

numBuffers - number of buffers in the batch

Returns:

minimum size of the compressed output sizes buffer needed

compressAsync

public static void compressAsync(HostMemoryBuffer compressedSizesOutputBuffer,
                                 BaseDeviceMemoryBuffer[] inputs,
                                 long chunkSize,
                                 BaseDeviceMemoryBuffer tempBuffer,
                                 BaseDeviceMemoryBuffer[] outputs,
                                 Cuda.Stream stream)

Asynchronously compress a batch of input buffers. The compressed size output buffer must be pinned memory for this operation to be truly asynchronous. Note that the caller must synchronize on the specified CUDA stream in order to safely examine the compressed output sizes!

Parameters:

compressedSizesOutputBuffer - host memory where the compressed output size will be stored

inputs - buffers to compress

chunkSize - type of data within each buffer

tempBuffer - compression chunk size to use

outputs - output buffers that will contain the compressed results

stream - CUDA stream to use

compress

public static BatchedLZ4Compressor.BatchedCompressionResult compress(BaseDeviceMemoryBuffer[] inputs,
                                                                     long chunkSize,
                                                                     Cuda.Stream stream)

Compress a batch of buffers with LZ4

Parameters:

inputs - buffers to compress

chunkSize - compression chunk size to use

stream - CUDA stream to use

Returns:

compression results containing the corresponding output buffer and compressed data size for each input buffer