org.apache.hudi.common.util

Class CleanerUtils

java.lang.Object

org.apache.hudi.common.util.CleanerUtils

public class CleanerUtils
extends Object

Utils for clean action.

Field Summary

Fields

Modifier and Type	Field and Description
static Integer	CLEAN_METADATA_VERSION_1
static Integer	CLEAN_METADATA_VERSION_2
static Integer	LATEST_CLEAN_METADATA_VERSION

Constructor Summary

Constructors

Constructor and Description
CleanerUtils()

Method Summary

Modifier and Type	Method and Description
static HoodieCleanMetadata	convertCleanMetadata(String startCleanTime, Option<Long> durationInMs, List<HoodieCleanStat> cleanStats, Map<String,String> extraMetadatafromCleanPlan)
static List<HoodieCleanFileInfo>	convertToHoodieCleanFileInfoList(List<CleanFileInfo> cleanFileInfoList) Convert list of cleanFileInfo instances to list of avro-generated HoodieCleanFileInfo instances.
static HoodieCleanMetadata	getCleanerMetadata(HoodieTableMetaClient metaClient, byte[] details) Get Latest Version of Hoodie Cleaner Metadata - Output of cleaner operation.
static HoodieCleanMetadata	getCleanerMetadata(HoodieTableMetaClient metaClient, HoodieInstant cleanInstant) Get Latest Version of Hoodie Cleaner Metadata - Output of cleaner operation.
static HoodieCleanerPlan	getCleanerPlan(HoodieTableMetaClient metaClient, byte[] details) Get Latest version of cleaner plan corresponding to a clean instant.
static HoodieCleanerPlan	getCleanerPlan(HoodieTableMetaClient metaClient, HoodieInstant cleanInstant) Get Latest version of cleaner plan corresponding to a clean instant.
static Option<HoodieInstant>	getEarliestCommitToRetain(HoodieTimeline commitsTimeline, HoodieCleaningPolicy cleaningPolicy, int commitsRetained, Instant latestInstant, int hoursRetained, HoodieTimelineTimeZone timeZone)
static void	rollbackFailedWrites(HoodieFailedWritesCleaningPolicy cleaningPolicy, String actionType, Functions.Function0<Boolean> rollbackFailedWritesFunc) Execute HoodieFailedWritesCleaningPolicy to rollback failed writes for different actions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLEAN_METADATA_VERSION_1

public static final Integer CLEAN_METADATA_VERSION_1

CLEAN_METADATA_VERSION_2

public static final Integer CLEAN_METADATA_VERSION_2

LATEST_CLEAN_METADATA_VERSION

public static final Integer LATEST_CLEAN_METADATA_VERSION

Constructor Detail

CleanerUtils

public CleanerUtils()

Method Detail

convertCleanMetadata

public static HoodieCleanMetadata convertCleanMetadata(String startCleanTime,
                                                       Option<Long> durationInMs,
                                                       List<HoodieCleanStat> cleanStats,
                                                       Map<String,String> extraMetadatafromCleanPlan)

getCleanerMetadata

public static HoodieCleanMetadata getCleanerMetadata(HoodieTableMetaClient metaClient,
                                                     HoodieInstant cleanInstant)
                                              throws IOException

Get Latest Version of Hoodie Cleaner Metadata - Output of cleaner operation.

Parameters:

metaClient - Hoodie Table Meta Client

cleanInstant - Instant referring to clean action

Returns:

Latest version of Clean metadata corresponding to clean instant

Throws:

IOException

getCleanerMetadata

public static HoodieCleanMetadata getCleanerMetadata(HoodieTableMetaClient metaClient,
                                                     byte[] details)
                                              throws IOException

Get Latest Version of Hoodie Cleaner Metadata - Output of cleaner operation.

Returns:

Latest version of Clean metadata corresponding to clean instant

Throws:

IOException

getEarliestCommitToRetain

public static Option<HoodieInstant> getEarliestCommitToRetain(HoodieTimeline commitsTimeline,
                                                              HoodieCleaningPolicy cleaningPolicy,
                                                              int commitsRetained,
                                                              Instant latestInstant,
                                                              int hoursRetained,
                                                              HoodieTimelineTimeZone timeZone)

getCleanerPlan

public static HoodieCleanerPlan getCleanerPlan(HoodieTableMetaClient metaClient,
                                               HoodieInstant cleanInstant)
                                        throws IOException

Get Latest version of cleaner plan corresponding to a clean instant.

Parameters:

metaClient - Hoodie Table Meta Client

cleanInstant - Instant referring to clean action

Returns:

Cleaner plan corresponding to clean instant

Throws:

IOException

getCleanerPlan

public static HoodieCleanerPlan getCleanerPlan(HoodieTableMetaClient metaClient,
                                               byte[] details)
                                        throws IOException

Get Latest version of cleaner plan corresponding to a clean instant.

Parameters:

metaClient - Hoodie Table Meta Client

Returns:

Cleaner plan corresponding to clean instant

Throws:

IOException

convertToHoodieCleanFileInfoList

public static List<HoodieCleanFileInfo> convertToHoodieCleanFileInfoList(List<CleanFileInfo> cleanFileInfoList)

Convert list of cleanFileInfo instances to list of avro-generated HoodieCleanFileInfo instances.

Parameters:

cleanFileInfoList -

Returns:

rollbackFailedWrites

public static void rollbackFailedWrites(HoodieFailedWritesCleaningPolicy cleaningPolicy,
                                        String actionType,
                                        Functions.Function0<Boolean> rollbackFailedWritesFunc)

Execute HoodieFailedWritesCleaningPolicy to rollback failed writes for different actions.

Parameters:

cleaningPolicy -

actionType -

rollbackFailedWritesFunc -