org.apache.hudi.common.table.timeline

Class TimelineUtils

java.lang.Object

org.apache.hudi.common.table.timeline.TimelineUtils

public class TimelineUtils
extends Object

TimelineUtils provides a common way to query incremental meta-data changes for a hoodie table. This is useful in multiple places including: 1) HiveSync - this can be used to query partitions that changed since previous sync. 2) Incremental reads - InputFormats can use this API to query

Constructor Summary

Constructors

Constructor and Description
TimelineUtils()

Method Summary

Modifier and Type	Method and Description
static List<String>	getAffectedPartitions(HoodieTimeline timeline) Returns partitions that have been modified including internal operations such as clean in the passed timeline.
static Map<String,Option<String>>	getAllExtraMetadataForKey(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from all active commit/deltacommit instants.
static List<String>	getDroppedPartitions(HoodieTimeline timeline) Returns partitions that have been deleted or marked for deletion in the given timeline.
static Option<String>	getExtraMetadataFromLatest(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from latest commit/deltacommit/replacecommit(eg.
static Option<String>	getExtraMetadataFromLatestIncludeClustering(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from latest commit/deltacommit/replacecommit instant including internal commits such as clustering.
static HoodieDefaultTimeline	getTimeline(HoodieTableMetaClient metaClient, boolean includeArchivedTimeline)
static List<String>	getWrittenPartitions(HoodieTimeline timeline) Returns partitions that have new data strictly after commitTime.
static boolean	isClusteringCommit(HoodieTableMetaClient metaClient, HoodieInstant instant)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TimelineUtils

public TimelineUtils()

Method Detail

getWrittenPartitions

public static List<String> getWrittenPartitions(HoodieTimeline timeline)

Returns partitions that have new data strictly after commitTime. Does not include internal operations such as clean in the timeline.

getDroppedPartitions

public static List<String> getDroppedPartitions(HoodieTimeline timeline)

Returns partitions that have been deleted or marked for deletion in the given timeline. Does not include internal operations such as clean in the timeline.

getAffectedPartitions

public static List<String> getAffectedPartitions(HoodieTimeline timeline)

Returns partitions that have been modified including internal operations such as clean in the passed timeline.

getExtraMetadataFromLatest

public static Option<String> getExtraMetadataFromLatest(HoodieTableMetaClient metaClient,
                                                        String extraMetadataKey)

Get extra metadata for specified key from latest commit/deltacommit/replacecommit(eg. insert_overwrite) instant.

getExtraMetadataFromLatestIncludeClustering

public static Option<String> getExtraMetadataFromLatestIncludeClustering(HoodieTableMetaClient metaClient,
                                                                         String extraMetadataKey)

Get extra metadata for specified key from latest commit/deltacommit/replacecommit instant including internal commits such as clustering.

getAllExtraMetadataForKey

public static Map<String,Option<String>> getAllExtraMetadataForKey(HoodieTableMetaClient metaClient,
                                                                   String extraMetadataKey)

Get extra metadata for specified key from all active commit/deltacommit instants.

isClusteringCommit

public static boolean isClusteringCommit(HoodieTableMetaClient metaClient,
                                         HoodieInstant instant)

getTimeline

public static HoodieDefaultTimeline getTimeline(HoodieTableMetaClient metaClient,
                                                boolean includeArchivedTimeline)