org.apache.hudi.client.clustering.plan.strategy

Class JavaSizeBasedClusteringPlanStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>

java.lang.Object

org.apache.hudi.table.action.cluster.strategy.ClusteringPlanStrategy<T,I,K,O>

org.apache.hudi.table.action.cluster.strategy.PartitionAwareClusteringPlanStrategy<T,List<org.apache.hudi.common.model.HoodieRecord<T>>,List<org.apache.hudi.common.model.HoodieKey>,List<WriteStatus>>

org.apache.hudi.client.clustering.plan.strategy.JavaSizeBasedClusteringPlanStrategy<T>

All Implemented Interfaces:

Serializable

public class JavaSizeBasedClusteringPlanStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>
extends PartitionAwareClusteringPlanStrategy<T,List<org.apache.hudi.common.model.HoodieRecord<T>>,List<org.apache.hudi.common.model.HoodieKey>,List<WriteStatus>>

Clustering Strategy for Java engine based on following. 1) Creates clustering groups based on max size allowed per group. 2) Excludes files that are greater than 'small.file.limit' from clustering plan.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.table.action.cluster.strategy.ClusteringPlanStrategy

CLUSTERING_PLAN_VERSION_1

Constructor Summary

Constructors

Constructor and Description
JavaSizeBasedClusteringPlanStrategy(HoodieJavaCopyOnWriteTable<T> table, HoodieJavaEngineContext engineContext, HoodieWriteConfig writeConfig)
JavaSizeBasedClusteringPlanStrategy(HoodieJavaMergeOnReadTable<T> table, HoodieJavaEngineContext engineContext, HoodieWriteConfig writeConfig)

Method Summary

Modifier and Type	Method and Description
protected Stream<org.apache.hudi.avro.model.HoodieClusteringGroup>	buildClusteringGroupsForPartition(String partitionPath, List<org.apache.hudi.common.model.FileSlice> fileSlices)
protected Stream<org.apache.hudi.common.model.FileSlice>	getFileSlicesEligibleForClustering(String partition)
protected Map<String,String>	getStrategyParams()

Methods inherited from class org.apache.hudi.table.action.cluster.strategy.PartitionAwareClusteringPlanStrategy

filterPartitionPaths, generateClusteringPlan, getMatchedPartitions, getRegexPatternMatchedPartitions

Methods inherited from class org.apache.hudi.table.action.cluster.strategy.ClusteringPlanStrategy

buildMetrics, checkAndGetClusteringPlanStrategy, getEngineContext, getExtraMetadata, getFileSliceInfo, getHoodieTable, getPlanVersion, getWriteConfig

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JavaSizeBasedClusteringPlanStrategy

public JavaSizeBasedClusteringPlanStrategy(HoodieJavaCopyOnWriteTable<T> table,
                                           HoodieJavaEngineContext engineContext,
                                           HoodieWriteConfig writeConfig)

JavaSizeBasedClusteringPlanStrategy

public JavaSizeBasedClusteringPlanStrategy(HoodieJavaMergeOnReadTable<T> table,
                                           HoodieJavaEngineContext engineContext,
                                           HoodieWriteConfig writeConfig)

Method Detail

buildClusteringGroupsForPartition

protected Stream<org.apache.hudi.avro.model.HoodieClusteringGroup> buildClusteringGroupsForPartition(String partitionPath,
                                                                                                     List<org.apache.hudi.common.model.FileSlice> fileSlices)

Specified by:

buildClusteringGroupsForPartition in class PartitionAwareClusteringPlanStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>,List<org.apache.hudi.common.model.HoodieRecord<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>>,List<org.apache.hudi.common.model.HoodieKey>,List<WriteStatus>>

getStrategyParams

protected Map<String,String> getStrategyParams()

Specified by:

getStrategyParams in class ClusteringPlanStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>,List<org.apache.hudi.common.model.HoodieRecord<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>>,List<org.apache.hudi.common.model.HoodieKey>,List<WriteStatus>>

getFileSlicesEligibleForClustering

protected Stream<org.apache.hudi.common.model.FileSlice> getFileSlicesEligibleForClustering(String partition)

Overrides:

getFileSlicesEligibleForClustering in class ClusteringPlanStrategy<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>,List<org.apache.hudi.common.model.HoodieRecord<T extends org.apache.hudi.common.model.HoodieRecordPayload<T>>>,List<org.apache.hudi.common.model.HoodieKey>,List<WriteStatus>>