org.apache.hudi.common.table.view

Class IncrementalTimelineSyncFileSystemView

java.lang.Object

org.apache.hudi.common.table.view.AbstractTableFileSystemView

org.apache.hudi.common.table.view.IncrementalTimelineSyncFileSystemView

All Implemented Interfaces:

Serializable, SyncableFileSystemView, TableFileSystemView, TableFileSystemView.BaseFileOnlyView, TableFileSystemView.BaseFileOnlyViewWithLatestSlice, TableFileSystemView.SliceView, TableFileSystemView.SliceViewWithLatestSlice

Direct Known Subclasses:

HoodieTableFileSystemView, RocksDbBasedFileSystemView

public abstract class IncrementalTimelineSyncFileSystemView
extends AbstractTableFileSystemView

Adds the capability to incrementally sync the changes to file-system view as and when new instants gets completed.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hudi.common.table.view.TableFileSystemView

TableFileSystemView.BaseFileOnlyView, TableFileSystemView.BaseFileOnlyViewWithLatestSlice, TableFileSystemView.SliceView, TableFileSystemView.SliceViewWithLatestSlice

Field Summary

Fields inherited from class org.apache.hudi.common.table.view.AbstractTableFileSystemView

metaClient

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	IncrementalTimelineSyncFileSystemView(boolean enableIncrementalTimelineSync)

Method Summary

Modifier and Type	Method and Description
protected void	applyDeltaFileSlicesToPartitionView(String partition, List<HoodieFileGroup> deltaFileGroups, org.apache.hudi.common.table.view.IncrementalTimelineSyncFileSystemView.DeltaApplyMode mode) Apply changes to partition file-system view.
HoodieTimeline	getTimeline() Timeline corresponding to the view.
protected void	refreshTimeline(HoodieTimeline visibleActiveTimeline) Refresh commits timeline.
protected void	runSync(HoodieTimeline oldTimeline, HoodieTimeline newTimeline) Performs complete reset of file-system view.

Methods inherited from class org.apache.hudi.common.table.view.AbstractTableFileSystemView

addBootstrapBaseFileIfPresent, addBootstrapBaseFileIfPresent, addBootstrapBaseFileIfPresent, addFilesToView, addReplacedFileGroups, buildFileGroups, buildFileGroups, close, fetchFileGroupsInPendingClustering, fetchLatestBaseFile, fetchLatestBaseFiles, fetchLatestFileSlice, filterBaseFileAfterPendingCompaction, getAllBaseFiles, getAllFileGroups, getAllFileSlices, getAllReplacedFileGroups, getBaseFileOn, getBootstrapBaseFile, getFileGroupsInPendingClustering, getLastInstant, getLatestBaseFile, getLatestBaseFile, getLatestBaseFiles, getLatestBaseFiles, getLatestBaseFilesBeforeOrOn, getLatestBaseFilesInRange, getLatestFileSlice, getLatestFileSliceInRange, getLatestFileSlices, getLatestFileSlicesBeforeOrOn, getLatestMergedFileSlicesBeforeOrOn, getLatestUnCompactedFileSlices, getPartitionPaths, getPendingClusteringInstant, getPendingCompactionOperations, getPendingCompactionOperationWithInstant, getReplacedFileGroupsBefore, getReplacedFileGroupsBeforeOrOn, getReplaceInstant, getVisibleCommitsAndCompactionTimeline, init, isBaseFileDueToPendingClustering, isBaseFileDueToPendingCompaction, isBootstrapBaseFilePresentForFileId, isFileSliceAfterPendingCompaction, isPendingClusteringScheduledForFileId, isPendingCompactionScheduledForFileId, listPartition, removeReplacedFileIdsAtInstants, reset, resetReplacedFileGroups, resetViewState, sync

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IncrementalTimelineSyncFileSystemView

protected IncrementalTimelineSyncFileSystemView(boolean enableIncrementalTimelineSync)

Method Detail

refreshTimeline

protected void refreshTimeline(HoodieTimeline visibleActiveTimeline)

Description copied from class: AbstractTableFileSystemView

Refresh commits timeline.

Overrides:

refreshTimeline in class AbstractTableFileSystemView

Parameters:

visibleActiveTimeline - Visible Active Timeline

runSync

protected void runSync(HoodieTimeline oldTimeline,
                       HoodieTimeline newTimeline)

Description copied from class: AbstractTableFileSystemView

Performs complete reset of file-system view. Subsequent partition view calls will load file slices against latest timeline

Overrides:

runSync in class AbstractTableFileSystemView

Parameters:

oldTimeline - Old Hoodie Timeline

newTimeline - New Hoodie Timeline

applyDeltaFileSlicesToPartitionView

protected void applyDeltaFileSlicesToPartitionView(String partition,
                                                   List<HoodieFileGroup> deltaFileGroups,
                                                   org.apache.hudi.common.table.view.IncrementalTimelineSyncFileSystemView.DeltaApplyMode mode)

Apply changes to partition file-system view. Base Implementation overwrites the entire partitions view assuming some sort of map (in-mem/disk-based) is used. For View implementation which supports fine-granular updates (e:g RocksDB), override this method.

Parameters:

partition - PartitionPath

deltaFileGroups - Changed file-slices aggregated as file-groups

mode - Delta Apply mode

getTimeline

public HoodieTimeline getTimeline()

Description copied from interface: TableFileSystemView

Timeline corresponding to the view.

Specified by:

getTimeline in interface TableFileSystemView

Overrides:

getTimeline in class AbstractTableFileSystemView