org.apache.hudi.table

Interface BulkInsertPartitioner<I>

All Superinterfaces:

Serializable

public interface BulkInsertPartitioner<I>
extends Serializable

Repartition input records into at least expected number of output partitions. It should give below guarantees - Output partition will have records from only one hoodie partition. - Average records per output partitions should be almost equal to (#inputRecords / #outputPartitions) to avoid possible skews.

Method Summary

Modifier and Type	Method and Description
boolean	arePartitionRecordsSorted()
default String	getFileIdPfx(int partitionId) Return file group id prefix for the given data partition.
default Option<WriteHandleFactory>	getWriteHandleFactory(int partitionId) Return write handle factory for the given partition.
I	repartitionRecords(I records, int outputPartitions) Repartitions the input records into at least expected number of output partitions.

Method Detail

repartitionRecords

I repartitionRecords(I records,
                     int outputPartitions)

Repartitions the input records into at least expected number of output partitions.

Parameters:

records - Input Hoodie records

outputPartitions - Expected number of output partitions

Returns:

arePartitionRecordsSorted

boolean arePartitionRecordsSorted()

Returns:

true if the records within a partition are sorted; false otherwise.

getFileIdPfx

default String getFileIdPfx(int partitionId)

Return file group id prefix for the given data partition. By defauult, return a new file group id prefix, so that incoming records will route to a fresh new file group

Parameters:

partitionId - data partition

Returns:

getWriteHandleFactory

default Option<WriteHandleFactory> getWriteHandleFactory(int partitionId)

Return write handle factory for the given partition.

Parameters:

partitionId - data partition

Returns: