org.apache.hudi.client

Class BaseHoodieClient

java.lang.Object

org.apache.hudi.client.BaseHoodieClient

All Implemented Interfaces:

Serializable, AutoCloseable

Direct Known Subclasses:

BaseHoodieTableServiceClient, BaseHoodieWriteClient, CompactionAdminClient

public abstract class BaseHoodieClient
extends Object
implements Serializable, AutoCloseable

Abstract class taking care of holding common member variables (FileSystem, SparkContext, HoodieConfigs) Also, manages embedded timeline-server if enabled.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected String	basePath
protected HoodieWriteConfig	config
protected HoodieEngineContext	context
protected org.apache.hadoop.conf.Configuration	hadoopConf
protected HoodieHeartbeatClient	heartbeatClient
protected HoodieMetrics	metrics
protected HoodieStorage	storage
protected TransactionManager	txnManager

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	BaseHoodieClient(HoodieEngineContext context, HoodieWriteConfig clientConfig)
protected	BaseHoodieClient(HoodieEngineContext context, HoodieWriteConfig clientConfig, Option<EmbeddedTimelineService> timelineServer)

Method Summary

Modifier and Type	Method and Description
void	close() Releases any resources used by the client.
protected HoodieTableMetaClient	createMetaClient(boolean loadActiveTimelineOnLoad)
protected void	finalizeWrite(HoodieTable table, String instantTime, List<HoodieWriteStat> stats) Finalize Write operation.
HoodieWriteConfig	getConfig()
HoodieEngineContext	getEngineContext()
HoodieHeartbeatClient	getHeartbeatClient()
Option<EmbeddedTimelineService>	getTimelineServer()
protected void	initWrapperFSMetrics()
protected void	resolveWriteConflict(HoodieTable table, HoodieCommitMetadata metadata, Set<String> pendingInflightAndRequestedInstants) Resolve write conflicts before commit.
protected void	writeTableMetadata(HoodieTable table, String instantTime, HoodieCommitMetadata metadata, HoodieData<WriteStatus> writeStatuses) Write the HoodieCommitMetadata to metadata table if available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

storage

protected final transient HoodieStorage storage

context

protected final transient HoodieEngineContext context

hadoopConf

protected final transient org.apache.hadoop.conf.Configuration hadoopConf

metrics

protected final transient HoodieMetrics metrics

config

protected final HoodieWriteConfig config

basePath

protected final String basePath

heartbeatClient

protected final HoodieHeartbeatClient heartbeatClient

txnManager

protected final TransactionManager txnManager

Constructor Detail

BaseHoodieClient

protected BaseHoodieClient(HoodieEngineContext context,
                           HoodieWriteConfig clientConfig)

BaseHoodieClient

protected BaseHoodieClient(HoodieEngineContext context,
                           HoodieWriteConfig clientConfig,
                           Option<EmbeddedTimelineService> timelineServer)

Method Detail

close

public void close()

Releases any resources used by the client.

Specified by:

close in interface AutoCloseable

getConfig

public HoodieWriteConfig getConfig()

getEngineContext

public HoodieEngineContext getEngineContext()

initWrapperFSMetrics

protected void initWrapperFSMetrics()

createMetaClient

protected HoodieTableMetaClient createMetaClient(boolean loadActiveTimelineOnLoad)

getTimelineServer

public Option<EmbeddedTimelineService> getTimelineServer()

getHeartbeatClient

public HoodieHeartbeatClient getHeartbeatClient()

resolveWriteConflict

protected void resolveWriteConflict(HoodieTable table,
                                    HoodieCommitMetadata metadata,
                                    Set<String> pendingInflightAndRequestedInstants)

Resolve write conflicts before commit.

Parameters:

table - A hoodie table instance created after transaction starts so that the latest commits and files are captured.

metadata - Current committing instant's metadata

pendingInflightAndRequestedInstants - Pending instants on the timeline

See Also:

BaseHoodieWriteClient#preCommit}, BaseHoodieTableServiceClient#preCommit}

finalizeWrite

protected void finalizeWrite(HoodieTable table,
                             String instantTime,
                             List<HoodieWriteStat> stats)

Finalize Write operation.

Parameters:

table - HoodieTable

instantTime - Instant Time

stats - Hoodie Write Stat

writeTableMetadata

protected void writeTableMetadata(HoodieTable table,
                                  String instantTime,
                                  HoodieCommitMetadata metadata,
                                  HoodieData<WriteStatus> writeStatuses)

Write the HoodieCommitMetadata to metadata table if available.

Parameters:

table - HoodieTable of interest.

instantTime - instant time of the commit.

metadata - instance of HoodieCommitMetadata.

writeStatuses - Write statuses of the commit