org.apache.hudi.client.clustering.run.strategy

Class JavaSortAndSizeExecutionStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>

java.lang.Object

org.apache.hudi.table.action.cluster.strategy.ClusteringExecutionStrategy<T,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieRecord<T>>,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieKey>,org.apache.hudi.common.data.HoodieData<WriteStatus>>

org.apache.hudi.client.clustering.run.strategy.JavaExecutionStrategy<T>

org.apache.hudi.client.clustering.run.strategy.JavaSortAndSizeExecutionStrategy<T>

All Implemented Interfaces:

Serializable

public class JavaSortAndSizeExecutionStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>
extends JavaExecutionStrategy<T>

Clustering Strategy based on following. 1) Java execution engine. 2) Uses bulk_insert to write data into new files.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
JavaSortAndSizeExecutionStrategy(HoodieTable table, org.apache.hudi.common.engine.HoodieEngineContext engineContext, HoodieWriteConfig writeConfig)

Method Summary

Modifier and Type

Method and Description

List<WriteStatus>

performClusteringWithRecordList(List<org.apache.hudi.common.model.HoodieRecord<T>> inputRecords, int numOutputGroups, String instantTime, Map<String,String> strategyParams, org.apache.avro.Schema schema, List<org.apache.hudi.common.model.HoodieFileGroupId> fileGroupIdList, boolean preserveHoodieMetadata) Execute clustering to write inputRecords into new files as defined by rules in strategy parameters.

Methods inherited from class org.apache.hudi.client.clustering.run.strategy.JavaExecutionStrategy

getPartitioner, performClustering

Methods inherited from class org.apache.hudi.table.action.cluster.strategy.ClusteringExecutionStrategy

getEngineContext, getHoodieTable, getWriteConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JavaSortAndSizeExecutionStrategy

public JavaSortAndSizeExecutionStrategy(HoodieTable table,
                                        org.apache.hudi.common.engine.HoodieEngineContext engineContext,
                                        HoodieWriteConfig writeConfig)

Method Detail

performClusteringWithRecordList

public List<WriteStatus> performClusteringWithRecordList(List<org.apache.hudi.common.model.HoodieRecord<T>> inputRecords,
                                                         int numOutputGroups,
                                                         String instantTime,
                                                         Map<String,String> strategyParams,
                                                         org.apache.avro.Schema schema,
                                                         List<org.apache.hudi.common.model.HoodieFileGroupId> fileGroupIdList,
                                                         boolean preserveHoodieMetadata)

Description copied from class: JavaExecutionStrategy

Execute clustering to write inputRecords into new files as defined by rules in strategy parameters. The number of new file groups created is bounded by numOutputGroups. Note that commit is not done as part of strategy. commit is callers responsibility.

Specified by:

performClusteringWithRecordList in class JavaExecutionStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>

Parameters:

inputRecords - List of HoodieRecord.

numOutputGroups - Number of output file groups.

instantTime - Clustering (replace commit) instant time.

strategyParams - Strategy parameters containing columns to sort the data by when clustering.

schema - Schema of the data including metadata fields.

fileGroupIdList - File group id corresponding to each out group.

preserveHoodieMetadata - Whether to preserve commit metadata while clustering.

Returns:

List of WriteStatus.