org.apache.hudi.common.table.view

Class FileSystemViewManager

java.lang.Object

org.apache.hudi.common.table.view.FileSystemViewManager

public class FileSystemViewManager
extends Object

A container that can potentially hold one or more table's file-system views. There is one view for each table. This is a view built against a timeline containing completed actions. In an embedded timeline-server mode, this typically holds only one table's view. In a stand-alone server mode, this can hold more than one table's views. FileSystemView can be stored "locally" using the following storage mechanisms: a. In Memory b. Spillable Map c. RocksDB But there can be cases where the file-system view is managed remoted. For example : Embedded Timeline Server). In this case, the clients will configure a remote filesystem view client (RemoteHoodieTableFileSystemView) for the table which can connect to the remote file system view and fetch views. THere are 2 modes here : REMOTE_FIRST and REMOTE_ONLY REMOTE_FIRST : The file-system view implementation on client side will act as a remote proxy. In case, if there is problem (or exceptions) querying remote file-system view, a backup local file-system view(using either one of in-memory, spillable, rocksDB) is used to server file-system view queries REMOTE_ONLY : In this case, there is no backup local file-system view. If there is problem (or exceptions) querying remote file-system view, then the exceptions are percolated back to client. FileSystemViewManager is designed to encapsulate the file-system view storage from clients using the file-system view. FileSystemViewManager uses a factory to construct specific implementation of file-system view and passes it to clients for querying.

Method Summary

Modifier and Type	Method and Description
void	clearFileSystemView(String basePath) Drops reference to File-System Views.
void	close() Closes all views opened.
static HoodieTableFileSystemView	createInMemoryFileSystemView(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieMetadataConfig metadataConfig)
static HoodieTableFileSystemView	createInMemoryFileSystemViewWithTimeline(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, HoodieMetadataConfig metadataConfig, HoodieTimeline timeline)
static FileSystemViewManager	createViewManager(HoodieEngineContext context, HoodieMetadataConfig metadataConfig, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig)
static FileSystemViewManager	createViewManager(HoodieEngineContext context, HoodieMetadataConfig metadataConfig, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig, SerializableSupplier<HoodieTableMetadata> metadataSupplier) Main Factory method for building file-system views.
static FileSystemViewManager	createViewManager(HoodieEngineContext context, HoodieMetadataConfig metadataConfig, FileSystemViewStorageConfig config, HoodieCommonConfig commonConfig, String basePath)
SyncableFileSystemView	getFileSystemView(HoodieTableMetaClient metaClient) Main API to get the file-system view for the base-path.
SyncableFileSystemView	getFileSystemView(String basePath) Main API to get the file-system view for the base-path.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

clearFileSystemView

public void clearFileSystemView(String basePath)

Drops reference to File-System Views. Future calls to view results in creating a new view

Parameters:

basePath -

getFileSystemView

public SyncableFileSystemView getFileSystemView(String basePath)

Main API to get the file-system view for the base-path.

Parameters:

basePath -

Returns:

getFileSystemView

public SyncableFileSystemView getFileSystemView(HoodieTableMetaClient metaClient)

Main API to get the file-system view for the base-path.

Parameters:

metaClient - HoodieTableMetaClient

Returns:

close

public void close()

Closes all views opened.

createInMemoryFileSystemView

public static HoodieTableFileSystemView createInMemoryFileSystemView(HoodieEngineContext engineContext,
                                                                     HoodieTableMetaClient metaClient,
                                                                     HoodieMetadataConfig metadataConfig)

createInMemoryFileSystemViewWithTimeline

public static HoodieTableFileSystemView createInMemoryFileSystemViewWithTimeline(HoodieEngineContext engineContext,
                                                                                 HoodieTableMetaClient metaClient,
                                                                                 HoodieMetadataConfig metadataConfig,
                                                                                 HoodieTimeline timeline)

createViewManager

public static FileSystemViewManager createViewManager(HoodieEngineContext context,
                                                      HoodieMetadataConfig metadataConfig,
                                                      FileSystemViewStorageConfig config,
                                                      HoodieCommonConfig commonConfig)

createViewManager

public static FileSystemViewManager createViewManager(HoodieEngineContext context,
                                                      HoodieMetadataConfig metadataConfig,
                                                      FileSystemViewStorageConfig config,
                                                      HoodieCommonConfig commonConfig,
                                                      String basePath)

createViewManager

public static FileSystemViewManager createViewManager(HoodieEngineContext context,
                                                      HoodieMetadataConfig metadataConfig,
                                                      FileSystemViewStorageConfig config,
                                                      HoodieCommonConfig commonConfig,
                                                      SerializableSupplier<HoodieTableMetadata> metadataSupplier)

Main Factory method for building file-system views.