org.apache.hudi.metadata

Class HoodieMetadataFileSystemView

java.lang.Object

org.apache.hudi.common.table.view.AbstractTableFileSystemView

org.apache.hudi.common.table.view.IncrementalTimelineSyncFileSystemView

org.apache.hudi.common.table.view.HoodieTableFileSystemView

org.apache.hudi.metadata.HoodieMetadataFileSystemView

All Implemented Interfaces:

Serializable, SyncableFileSystemView, TableFileSystemView, TableFileSystemView.BaseFileOnlyView, TableFileSystemView.BaseFileOnlyViewWithLatestSlice, TableFileSystemView.SliceView, TableFileSystemView.SliceViewWithLatestSlice

public class HoodieMetadataFileSystemView
extends HoodieTableFileSystemView

HoodieTableFileSystemView implementation that retrieved partition listings from the Metadata Table.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hudi.common.table.view.TableFileSystemView

TableFileSystemView.BaseFileOnlyView, TableFileSystemView.BaseFileOnlyViewWithLatestSlice, TableFileSystemView.SliceView, TableFileSystemView.SliceViewWithLatestSlice

Field Summary

Fields inherited from class org.apache.hudi.common.table.view.HoodieTableFileSystemView

fgIdToBootstrapBaseFile, fgIdToPendingClustering, fgIdToPendingCompaction, fgIdToReplaceInstants, partitionToFileGroupsMap

Fields inherited from class org.apache.hudi.common.table.view.AbstractTableFileSystemView

metaClient

Constructor Summary

Constructors

Constructor and Description
HoodieMetadataFileSystemView(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieTimeline visibleActiveTimeline, HoodieMetadataConfig metadataConfig)
HoodieMetadataFileSystemView(HoodieTableMetaClient metaClient, HoodieTimeline visibleActiveTimeline, HoodieTableMetadata tableMetadata)

Method Summary

Modifier and Type	Method and Description
void	close() Allow View to release resources and close.
protected org.apache.hadoop.fs.FileStatus[]	listPartition(org.apache.hadoop.fs.Path partitionPath) Return all the files in the partition by reading from the Metadata Table.
void	reset() Clears the partition Map and reset view states.
void	sync() Read the latest timeline and refresh the file-system view to match the current state of the file-system.

Methods inherited from class org.apache.hudi.common.table.view.HoodieTableFileSystemView

addPendingCompactionOperations, addReplacedFileGroups, createFileIdToBootstrapBaseFileMap, createFileIdToPendingClusteringMap, createFileIdToPendingCompactionMap, createFileIdToReplaceInstantMap, createPartitionToFileGroups, fetchAllStoredFileGroups, fetchFileGroupsInPendingClustering, fetchLatestFileSlicesIncludingInflight, getAllFileGroups, getBootstrapBaseFile, getPendingClusteringInstant, getPendingCompactionOperationWithInstant, getReplaceInstant, init, init, isBootstrapBaseFilePresentForFileId, isClosed, isPartitionAvailableInStore, isPendingClusteringScheduledForFileId, isPendingCompactionScheduledForFileId, removePendingCompactionOperations, removeReplacedFileIdsAtInstants, resetPendingCompactionOperations, resetReplacedFileGroups, resetViewState, storePartitionView

Methods inherited from class org.apache.hudi.common.table.view.IncrementalTimelineSyncFileSystemView

applyDeltaFileSlicesToPartitionView, getTimeline, refreshTimeline, runSync

Methods inherited from class org.apache.hudi.common.table.view.AbstractTableFileSystemView

addBootstrapBaseFileIfPresent, addBootstrapBaseFileIfPresent, addBootstrapBaseFileIfPresent, addFilesToView, buildFileGroups, buildFileGroups, fetchLatestBaseFile, fetchLatestBaseFiles, fetchLatestFileSlice, filterBaseFileAfterPendingCompaction, getAllBaseFiles, getAllFileGroups, getAllFileSlices, getAllReplacedFileGroups, getBaseFileOn, getFileGroupsInPendingClustering, getLastInstant, getLatestBaseFile, getLatestBaseFile, getLatestBaseFiles, getLatestBaseFiles, getLatestBaseFilesBeforeOrOn, getLatestBaseFilesInRange, getLatestFileSlice, getLatestFileSliceInRange, getLatestFileSlices, getLatestFileSlicesBeforeOrOn, getLatestMergedFileSlicesBeforeOrOn, getLatestUnCompactedFileSlices, getPartitionPaths, getPendingCompactionOperations, getReplacedFileGroupsBefore, getReplacedFileGroupsBeforeOrOn, getVisibleCommitsAndCompactionTimeline, isBaseFileDueToPendingClustering, isBaseFileDueToPendingCompaction, isFileSliceAfterPendingCompaction

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieMetadataFileSystemView

public HoodieMetadataFileSystemView(HoodieTableMetaClient metaClient,
                                    HoodieTimeline visibleActiveTimeline,
                                    HoodieTableMetadata tableMetadata)

HoodieMetadataFileSystemView

public HoodieMetadataFileSystemView(HoodieEngineContext engineContext,
                                    HoodieTableMetaClient metaClient,
                                    HoodieTimeline visibleActiveTimeline,
                                    HoodieMetadataConfig metadataConfig)

Method Detail

listPartition

protected org.apache.hadoop.fs.FileStatus[] listPartition(org.apache.hadoop.fs.Path partitionPath)
                                                   throws IOException

Return all the files in the partition by reading from the Metadata Table.

Overrides:

listPartition in class AbstractTableFileSystemView

Parameters:

partitionPath - The absolute path of the partition

Throws:

IOException

reset

public void reset()

Description copied from class: AbstractTableFileSystemView

Clears the partition Map and reset view states.

Specified by:

reset in interface SyncableFileSystemView

Overrides:

reset in class AbstractTableFileSystemView

sync

public void sync()

Description copied from interface: SyncableFileSystemView

Read the latest timeline and refresh the file-system view to match the current state of the file-system. The refresh can either be done incrementally (from reading file-slices in metadata files) or from scratch by reseting view storage.

Specified by:

sync in interface SyncableFileSystemView

Overrides:

sync in class AbstractTableFileSystemView

close

public void close()

Description copied from interface: SyncableFileSystemView

Allow View to release resources and close.

Specified by:

close in interface SyncableFileSystemView

Overrides:

close in class HoodieTableFileSystemView