org.apache.hudi.client.clustering.run.strategy

Class JavaExecutionStrategy<T>

java.lang.Object

org.apache.hudi.table.action.cluster.strategy.ClusteringExecutionStrategy<T,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieRecord<T>>,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieKey>,org.apache.hudi.common.data.HoodieData<WriteStatus>>

org.apache.hudi.client.clustering.run.strategy.JavaExecutionStrategy<T>

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

JavaSortAndSizeExecutionStrategy

public abstract class JavaExecutionStrategy<T>
extends ClusteringExecutionStrategy<T,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieRecord<T>>,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieKey>,org.apache.hudi.common.data.HoodieData<WriteStatus>>

Clustering strategy for Java engine.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.table.action.cluster.strategy.ClusteringExecutionStrategy

recordType, writeConfig

Constructor Summary

Constructors

Constructor and Description
JavaExecutionStrategy(HoodieTable table, org.apache.hudi.common.engine.HoodieEngineContext engineContext, HoodieWriteConfig writeConfig)

Method Summary

Modifier and Type	Method and Description
protected BulkInsertPartitioner<List<org.apache.hudi.common.model.HoodieRecord<T>>>	getPartitioner(Map<String,String> strategyParams, org.apache.avro.Schema schema) Create BulkInsertPartitioner based on strategy params.
HoodieWriteMetadata<org.apache.hudi.common.data.HoodieData<WriteStatus>>	performClustering(org.apache.hudi.avro.model.HoodieClusteringPlan clusteringPlan, org.apache.avro.Schema schema, String instantTime)
abstract List<WriteStatus>	performClusteringWithRecordList(List<org.apache.hudi.common.model.HoodieRecord<T>> inputRecords, int numOutputGroups, String instantTime, Map<String,String> strategyParams, org.apache.avro.Schema schema, List<org.apache.hudi.common.model.HoodieFileGroupId> fileGroupIdList, boolean preserveHoodieMetadata) Execute clustering to write inputRecords into new files as defined by rules in strategy parameters.

Methods inherited from class org.apache.hudi.table.action.cluster.strategy.ClusteringExecutionStrategy

getEngineContext, getHoodieTable, getWriteConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JavaExecutionStrategy

public JavaExecutionStrategy(HoodieTable table,
                             org.apache.hudi.common.engine.HoodieEngineContext engineContext,
                             HoodieWriteConfig writeConfig)

Method Detail

performClustering

public HoodieWriteMetadata<org.apache.hudi.common.data.HoodieData<WriteStatus>> performClustering(org.apache.hudi.avro.model.HoodieClusteringPlan clusteringPlan,
                                                                                                  org.apache.avro.Schema schema,
                                                                                                  String instantTime)

Specified by:

performClustering in class ClusteringExecutionStrategy<T,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieRecord<T>>,org.apache.hudi.common.data.HoodieData<org.apache.hudi.common.model.HoodieKey>,org.apache.hudi.common.data.HoodieData<WriteStatus>>

performClusteringWithRecordList

public abstract List<WriteStatus> performClusteringWithRecordList(List<org.apache.hudi.common.model.HoodieRecord<T>> inputRecords,
                                                                  int numOutputGroups,
                                                                  String instantTime,
                                                                  Map<String,String> strategyParams,
                                                                  org.apache.avro.Schema schema,
                                                                  List<org.apache.hudi.common.model.HoodieFileGroupId> fileGroupIdList,
                                                                  boolean preserveHoodieMetadata)

Execute clustering to write inputRecords into new files as defined by rules in strategy parameters. The number of new file groups created is bounded by numOutputGroups. Note that commit is not done as part of strategy. commit is callers responsibility.

Parameters:

inputRecords - List of HoodieRecord.

numOutputGroups - Number of output file groups.

instantTime - Clustering (replace commit) instant time.

strategyParams - Strategy parameters containing columns to sort the data by when clustering.

schema - Schema of the data including metadata fields.

fileGroupIdList - File group id corresponding to each out group.

preserveHoodieMetadata - Whether to preserve commit metadata while clustering.

Returns:

List of WriteStatus.

getPartitioner

protected BulkInsertPartitioner<List<org.apache.hudi.common.model.HoodieRecord<T>>> getPartitioner(Map<String,String> strategyParams,
                                                                                                   org.apache.avro.Schema schema)

Create BulkInsertPartitioner based on strategy params.

Parameters:

strategyParams - Strategy parameters containing columns to sort the data by when clustering.

schema - Schema of the data including metadata fields.

Returns:

partitioner for the java engine