org.apache.hudi.table.action.commit

Class BaseBulkInsertHelper<T,I,K,O,R>

java.lang.Object

org.apache.hudi.table.action.commit.ParallelismHelper<I>

org.apache.hudi.table.action.commit.BaseBulkInsertHelper<T,I,K,O,R>

public abstract class BaseBulkInsertHelper<T,I,K,O,R>
extends ParallelismHelper<I>

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseBulkInsertHelper(SerializableFunctionUnchecked<I,Integer> partitionNumberExtractor)

Method Summary

Modifier and Type	Method and Description
abstract HoodieWriteMetadata<O>	bulkInsert(I inputRecords, String instantTime, HoodieTable<T,I,K,O> table, HoodieWriteConfig config, BaseCommitActionExecutor<T,I,K,O,R> executor, boolean performDedupe, Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner) Mark instant as inflight, write input records, update index and return result.
abstract O	bulkInsert(I inputRecords, String instantTime, HoodieTable<T,I,K,O> table, HoodieWriteConfig config, boolean performDedupe, BulkInsertPartitioner partitioner, boolean addMetadataFields, int parallelism, WriteHandleFactory writeHandleFactory) Only write input records.

Methods inherited from class org.apache.hudi.table.action.commit.ParallelismHelper

deduceShuffleParallelism

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseBulkInsertHelper

protected BaseBulkInsertHelper(SerializableFunctionUnchecked<I,Integer> partitionNumberExtractor)

Method Detail

bulkInsert

public abstract HoodieWriteMetadata<O> bulkInsert(I inputRecords,
                                                  String instantTime,
                                                  HoodieTable<T,I,K,O> table,
                                                  HoodieWriteConfig config,
                                                  BaseCommitActionExecutor<T,I,K,O,R> executor,
                                                  boolean performDedupe,
                                                  Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner)

Mark instant as inflight, write input records, update index and return result.

bulkInsert

public abstract O bulkInsert(I inputRecords,
                             String instantTime,
                             HoodieTable<T,I,K,O> table,
                             HoodieWriteConfig config,
                             boolean performDedupe,
                             BulkInsertPartitioner partitioner,
                             boolean addMetadataFields,
                             int parallelism,
                             WriteHandleFactory writeHandleFactory)

Only write input records. Does not change timeline/index. Return information about new files created.

Parameters:

writeHandleFactory - default write handle factory writing records.