org.apache.hudi.table

Interface BulkInsertPartitioner<I>

All Superinterfaces:

Serializable

All Known Implementing Classes:

BucketIndexBulkInsertPartitioner

public interface BulkInsertPartitioner<I>
extends Serializable

Partitions the input records for bulk insert operation.

The actual implementation of BulkInsertPartitioner is determined by the bulk insert sort mode, BulkInsertSortMode, specified by HoodieWriteConfig.BULK_INSERT_SORT_MODE (`hoodie.bulkinsert.sort.mode`).

Method Summary

Modifier and Type	Method and Description
boolean	arePartitionRecordsSorted()
default String	getFileIdPfx(int partitionId) Return file group id prefix for the given data partition.
default Option<WriteHandleFactory>	getWriteHandleFactory(int partitionId) Return write handle factory for the given partition.
I	repartitionRecords(I records, int outputPartitions) Partitions the input records based on the number of output partitions as a hint.
static String[]	tryPrependPartitionPathColumns(String[] columnNames, HoodieWriteConfig config)

Method Detail

repartitionRecords

I repartitionRecords(I records,
                     int outputPartitions)

Partitions the input records based on the number of output partitions as a hint.

Note that, the number of output partitions may or may not be enforced, depending on the specific implementation.

Parameters:

records - Input Hoodie records.

outputPartitions - Expected number of output partitions as a hint.

Returns:

arePartitionRecordsSorted

boolean arePartitionRecordsSorted()

Returns:

true if the records are sorted by partition-path; false otherwise.

getFileIdPfx

default String getFileIdPfx(int partitionId)

Return file group id prefix for the given data partition. By default, return a new file group id prefix, so that incoming records will route to a fresh new file group

Parameters:

partitionId - data partition

Returns:

getWriteHandleFactory

default Option<WriteHandleFactory> getWriteHandleFactory(int partitionId)

Return write handle factory for the given partition.

Parameters:

partitionId - data partition

Returns:

tryPrependPartitionPathColumns

static String[] tryPrependPartitionPathColumns(String[] columnNames,
                                               HoodieWriteConfig config)