BaseHoodieWriteClient (hudi-client-common 0.14.0 API)

java.lang.Object
- org.apache.hudi.client.BaseHoodieClient
- - org.apache.hudi.client.BaseHoodieWriteClient<T,I,K,O>

Type Parameters:

T - Type of data

I - Type of inputs

K - Type of keys

O - Type of outputs

All Implemented Interfaces:

Serializable, AutoCloseable, RunsTableService
```
public abstract class BaseHoodieWriteClient<T,I,K,O>
extends BaseHoodieClient
implements RunsTableService
```
Abstract Write Client providing functionality for performing commit, index updates and rollback Reused for regular write operations like upsert/insert/bulk-insert as well as bootstrap

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected Option<Pair<HoodieInstant,Map<String,String>>>`	`lastCompletedTxnAndMetadata`
`protected static String`	`LOOKUP_STR`
`protected Set<String>`	`pendingInflightAndRequestedInstants`
`protected BaseHoodieTableServiceClient<?,?,O>`	`tableServiceClient`
`protected com.codahale.metrics.Timer.Context`	`writeTimer`

Fields inherited from class org.apache.hudi.client.BaseHoodieClient
basePath, config, context, fs, hadoopConf, heartbeatClient, metrics, txnManager

Constructor Summary

Constructors
Constructor and Description
`BaseHoodieWriteClient(HoodieEngineContext context, HoodieWriteConfig writeConfig, Option<EmbeddedTimelineService> timelineService, SupportsUpgradeDowngrade upgradeDowngradeHelper)` Deprecated.
`BaseHoodieWriteClient(HoodieEngineContext context, HoodieWriteConfig writeConfig, SupportsUpgradeDowngrade upgradeDowngradeHelper)` Deprecated.

Method Summary

All Methods Instance Methods Abstract Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`void`	`addColumn(String colName, org.apache.avro.Schema schema)`
`void`	`addColumn(String colName, org.apache.avro.Schema schema, String doc, String position, TableChange.ColumnPositionChange.ColumnPositionType positionType)` add columns to table.
`void`	`archive()` Trigger archival for the table.
`protected void`	`archive(HoodieTable table)` Trigger archival for the table.
`protected void`	`autoArchiveOnCommit(HoodieTable table)`
`protected void`	`autoCleanOnCommit()`
`void`	`bootstrap(Option<Map<String,String>> extraMetadata)` Main API to run bootstrap to hudi.
`abstract O`	`bulkInsert(I records, String instantTime)` Loads the given HoodieRecords, as inserts into the table.
`abstract O`	`bulkInsert(I records, String instantTime, Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner)` Loads the given HoodieRecords, as inserts into the table.
`abstract O`	`bulkInsertPreppedRecords(I preppedRecords, String instantTime, Option<BulkInsertPartitioner> bulkInsertPartitioner)` Loads the given HoodieRecords, as inserts into the table.
`HoodieCleanMetadata`	`clean()`
`HoodieCleanMetadata`	`clean(boolean skipLocking)` Deprecated.
`HoodieCleanMetadata`	`clean(String cleanInstantTime)` Clean up any stale/old files/data lying around (either on file storage or index storage) based on the configurations and CleaningPolicy used.
`HoodieCleanMetadata`	`clean(String cleanInstantTime, boolean skipLocking)` Deprecated.
`HoodieCleanMetadata`	`clean(String cleanInstantTime, boolean scheduleInline, boolean skipLocking)` Clean up any stale/old files/data lying around (either on file storage or index storage) based on the configurations and CleaningPolicy used.
`void`	`close()` Releases any resources used by the client.
`HoodieWriteMetadata<O>`	`cluster(String clusteringInstantTime)` Performs Clustering for the workload stored in instant-time.
`HoodieWriteMetadata<O>`	`cluster(String clusteringInstant, boolean shouldComplete)` Ensures clustering instant is in expected state and performs clustering for the plan stored in metadata.
`protected void`	`commit(HoodieTable table, String commitActionType, String instantTime, HoodieCommitMetadata metadata, List<HoodieWriteStat> stats, HoodieData<WriteStatus> writeStatuses)`
`boolean`	`commit(String instantTime, O writeStatuses)` Commit changes performed at the given instantTime marker.
`boolean`	`commit(String instantTime, O writeStatuses, Option<Map<String,String>> extraMetadata)` Commit changes performed at the given instantTime marker.
`boolean`	`commit(String instantTime, O writeStatuses, Option<Map<String,String>> extraMetadata, String commitActionType, Map<String,List<String>> partitionToReplacedFileIds)`
`abstract boolean`	`commit(String instantTime, O writeStatuses, Option<Map<String,String>> extraMetadata, String commitActionType, Map<String,List<String>> partitionToReplacedFileIds, Option<BiConsumer<HoodieTableMetaClient,HoodieCommitMetadata>> extraPreCommitFunc)`
`void`	`commitCompaction(String compactionInstantTime, HoodieCommitMetadata metadata, Option<Map<String,String>> extraMetadata)` Commit a compaction operation.
`void`	`commitLogCompaction(String logCompactionInstantTime, HoodieCommitMetadata metadata, Option<Map<String,String>> extraMetadata)` Commit a log compaction operation.
`boolean`	`commitStats(String instantTime, HoodieData<WriteStatus> writeStatuses, List<HoodieWriteStat> stats, Option<Map<String,String>> extraMetadata, String commitActionType)`
`boolean`	`commitStats(String instantTime, HoodieData<WriteStatus> writeStatuses, List<HoodieWriteStat> stats, Option<Map<String,String>> extraMetadata, String commitActionType, Map<String,List<String>> partitionToReplaceFileIds, Option<BiConsumer<HoodieTableMetaClient,HoodieCommitMetadata>> extraPreCommitFunc)`
`HoodieWriteMetadata<O>`	`compact(String compactionInstantTime)` Performs Compaction for the workload stored in instant-time.
`protected HoodieWriteMetadata<O>`	`compact(String compactionInstantTime, boolean shouldComplete)` Ensures compaction instant is in expected state and performs Compaction for the workload stored in instant-time.
`protected void`	`completeCompaction(HoodieCommitMetadata metadata, HoodieTable table, String compactionCommitTime)` Commit Compaction and track metrics.
`protected void`	`completeLogCompaction(HoodieCommitMetadata metadata, HoodieTable table, String logCompactionCommitTime)` Commit Log Compaction and track metrics.
`protected abstract HoodieIndex<?,?>`	`createIndex(HoodieWriteConfig writeConfig)`
`protected abstract HoodieTable<T,I,K,O>`	`createTable(HoodieWriteConfig config, org.apache.hadoop.conf.Configuration hadoopConf)`
`protected abstract HoodieTable<T,I,K,O>`	`createTable(HoodieWriteConfig config, org.apache.hadoop.conf.Configuration hadoopConf, HoodieTableMetaClient metaClient)`
`abstract O`	`delete(K keys, String instantTime)` Deletes a list of `HoodieKey`s from the Hoodie table, at the supplied instantTime `HoodieKey`s will be de-duped and non-existent keys will be removed before deleting.
`void`	`deleteColumns(String... colNames)` delete columns to table.
`abstract O`	`deletePrepped(I preppedRecords, String instantTime)` Delete records from Hoodie table based on `HoodieKey` and `HoodieRecordLocation` specified in preppedRecords.
`void`	`deleteSavepoint()` Delete a savepoint based on the latest commit action on the savepoint timeline.
`void`	`deleteSavepoint(String savepointTime)` Delete a savepoint that was created.
`protected void`	`doInitTable(WriteOperationType operationType, HoodieTableMetaClient metaClient, Option<String> instantTime)` Performs necessary bootstrapping operations (for ex, validating whether Metadata Table has to be bootstrapped).
`void`	`dropIndex(List<MetadataPartitionType> partitionTypes)` Drops the index and removes the metadata partitions.
`abstract I`	`filterExists(I hoodieRecords)` Filter out HoodieRecords that already exists in the output folder.
`HoodieIndex<?,?>`	`getIndex()`
`HoodieMetrics`	`getMetrics()`
`WriteOperationType`	`getOperationType()`
`BaseHoodieTableServiceClient<?,?,O>`	`getTableServiceClient()`
`Option<HoodieIndexCommitMetadata>`	`index(String indexInstantTime)` Runs INDEX action to build out the metadata partitions as planned for the given instant time.
`protected void`	`initMetadataTable(Option<String> instantTime)` Bootstrap the metadata table.
`HoodieTable`	`initTable(WriteOperationType operationType, Option<String> instantTime)` Instantiates and initializes instance of `HoodieTable`, performing crucial bootstrapping operations such as: NOTE: This method is engine-agnostic and SHOULD NOT be overloaded, please check on `doInitTable(WriteOperationType, HoodieTableMetaClient, Option)` instead Checking whether upgrade/downgrade is required Bootstrapping Metadata Table (if required) Initializing metrics contexts
`protected HoodieTable`	`initTable(WriteOperationType operationType, Option<String> instantTime, boolean initMetadataTable)`
`protected Option<String>`	`inlineScheduleCompaction(Option<Map<String,String>> extraMetadata)` Schedules compaction inline.
`abstract O`	`insert(I records, String instantTime)` Inserts the given HoodieRecords, into the table.
`abstract O`	`insertPreppedRecords(I preppedRecords, String instantTime)` Inserts the given prepared records into the Hoodie table, at the supplied instantTime.
`boolean`	`lazyRollbackFailedIndexing()` Rolls back the failed delta commits corresponding to the indexing action.
`HoodieWriteMetadata<O>`	`logCompact(String logCompactionInstantTime)` Performs Log Compaction for the workload stored in instant-time.
`protected HoodieWriteMetadata<O>`	`logCompact(String logCompactionInstantTime, boolean shouldComplete)` Ensures compaction instant is in expected state and performs Log Compaction for the workload stored in instant-time.s
`protected void`	`mayBeCleanAndArchive(HoodieTable table)` Triggers cleaning and archival for the table of interest.
`protected void`	`postCommit(HoodieTable table, HoodieCommitMetadata metadata, String instantTime, Option<Map<String,String>> extraMetadata)` Post Commit Hook.
`O`	`postWrite(HoodieWriteMetadata<O> result, String instantTime, HoodieTable hoodieTable)` Common method containing steps to be performed after write (upsert/insert/..) operations including auto-commit.
`protected void`	`preCommit(HoodieInstant inflightInstant, HoodieCommitMetadata metadata)` Any pre-commit actions like conflict resolution goes here.
`void`	`preWrite(String instantTime, WriteOperationType writeOperationType, HoodieTableMetaClient metaClient)` Common method containing steps to be performed before write (upsert/insert/...
`protected void`	`releaseResources(String instantTime)` Called after each write, to release any resources used.
`void`	`renameColumn(String colName, String newName)` rename col name for hudi table.
`void`	`reOrderColPosition(String colName, String referColName, TableChange.ColumnPositionChange.ColumnPositionType orderType)` reorder the position of col.
`HoodieRestoreMetadata`	`restoreToInstant(String savepointToRestoreTimestamp, boolean initialMetadataTableIfNecessary)` NOTE : This action requires all writers (ingest and compact) to a table to be stopped before proceeding.
`void`	`restoreToSavepoint()` Restore the data to a savepoint based on the latest commit action on the savepoint timeline.
`void`	`restoreToSavepoint(String savepointTime)` Restore the data to the savepoint.
`boolean`	`rollback(String commitInstantTime)` Deprecated.
`boolean`	`rollback(String commitInstantTime, String rollbackInstantTimestamp)` Deprecated.
`boolean`	`rollbackFailedWrites()` Rollback failed writes if any.
`void`	`runAnyPendingCompactions()` Run any pending compactions.
`void`	`runAnyPendingLogCompactions()` Run any pending log compactions.
`protected void`	`runTableServicesInline(HoodieTable table, HoodieCommitMetadata metadata, Option<Map<String,String>> extraMetadata)`
`void`	`savepoint(String user, String comment)` Create a savepoint based on the latest commit action on the timeline.
`void`	`savepoint(String instantTime, String user, String comment)` Savepoint a specific commit instant time.
`protected boolean`	`scheduleCleaningAtInstant(String instantTime, Option<Map<String,String>> extraMetadata)` Schedules a new cleaning instant with passed-in instant time.
`Option<String>`	`scheduleClustering(Option<Map<String,String>> extraMetadata)` Schedules a new clustering instant.
`boolean`	`scheduleClusteringAtInstant(String instantTime, Option<Map<String,String>> extraMetadata)` Schedules a new clustering instant with passed-in instant time.
`Option<String>`	`scheduleCompaction(Option<Map<String,String>> extraMetadata)` Schedules a new compaction instant.
`boolean`	`scheduleCompactionAtInstant(String instantTime, Option<Map<String,String>> extraMetadata)` Schedules a new compaction instant with passed-in instant time.
`Option<String>`	`scheduleIndexing(List<MetadataPartitionType> partitionTypes)` Schedules INDEX action.
`Option<String>`	`scheduleLogCompaction(Option<Map<String,String>> extraMetadata)` Schedules a new log compaction instant.
`boolean`	`scheduleLogCompactionAtInstant(String instantTime, Option<Map<String,String>> extraMetadata)` Schedules a new log compaction instant with passed-in instant time.
`Option<String>`	`scheduleTableService(Option<Map<String,String>> extraMetadata, TableServiceType tableServiceType)` Schedule table services such as clustering, compaction & cleaning.
`Option<String>`	`scheduleTableService(String instantTime, Option<Map<String,String>> extraMetadata, TableServiceType tableServiceType)` Schedule table services such as clustering, compaction & cleaning.
`void`	`setOperationType(WriteOperationType operationType)`
`protected void`	`setWriteSchemaForDeletes(HoodieTableMetaClient metaClient)` Sets write schema from last instant since deletes may not have schema set in the config.
`String`	`startCommit()` Provides a new commit time for a write operation (insert/update/delete).
`String`	`startCommit(String actionType, HoodieTableMetaClient metaClient)` Provides a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) with specified action.
`void`	`startCommitWithTime(String instantTime)` Provides a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) without specified action.
`void`	`startCommitWithTime(String instantTime, String actionType)` Completes a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) with specified action.
`protected void`	`tryUpgrade(HoodieTableMetaClient metaClient, Option<String> instantTime)` Upgrades the hoodie table if need be when moving to a new Hudi version.
`void`	`updateColumnComment(String colName, String doc)` update col comment for hudi table.
`void`	`updateColumnNullability(String colName, boolean nullable)` update col nullable attribute for hudi table.
`void`	`updateColumnType(String colName, Type newType)` update col Type for hudi table.
`abstract O`	`upsert(I records, String instantTime)` Upsert a batch of new records into Hoodie table at the supplied instantTime.
`abstract O`	`upsertPreppedRecords(I preppedRecords, String instantTime)` Upserts the given prepared records into the Hoodie table, at the supplied instantTime.
`protected void`	`writeTableMetadata(HoodieTable table, String instantTime, HoodieCommitMetadata metadata, HoodieData<WriteStatus> writeStatuses)` Write the HoodieCommitMetadata to metadata table if available.

Methods inherited from class org.apache.hudi.client.BaseHoodieClient
createMetaClient, finalizeWrite, getConfig, getEngineContext, getHeartbeatClient, getTimelineServer, initWrapperFSMetrics, resolveWriteConflict

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.client.RunsTableService
shouldDelegateToTableServiceManager, tableServicesEnabled

- Field Detail
  - LOOKUP_STR
```
protected static final String LOOKUP_STR
```
    See Also:
    
    Constant Field Values
  - writeTimer
```
protected transient com.codahale.metrics.Timer.Context writeTimer
```
  - lastCompletedTxnAndMetadata
```
protected Option<Pair<HoodieInstant,Map<String,String>>> lastCompletedTxnAndMetadata
```
  - pendingInflightAndRequestedInstants
```
protected Set<String> pendingInflightAndRequestedInstants
```
  - tableServiceClient
```
protected BaseHoodieTableServiceClient<?,?,O> tableServiceClient
```
- Constructor Detail
  - BaseHoodieWriteClient
```
@Deprecated
public BaseHoodieWriteClient(HoodieEngineContext context,
                                         HoodieWriteConfig writeConfig,
                                         SupportsUpgradeDowngrade upgradeDowngradeHelper)
```
    Deprecated.
    
    Create a write client, with new hudi index.
    
    Parameters:
    
    context - HoodieEngineContext
    
    writeConfig - instance of HoodieWriteConfig
    
    upgradeDowngradeHelper - engine-specific instance of SupportsUpgradeDowngrade
  - BaseHoodieWriteClient
```
@Deprecated
public BaseHoodieWriteClient(HoodieEngineContext context,
                                         HoodieWriteConfig writeConfig,
                                         Option<EmbeddedTimelineService> timelineService,
                                         SupportsUpgradeDowngrade upgradeDowngradeHelper)
```
    Deprecated.
    
    Create a write client, allows to specify all parameters.
    
    Parameters:
    
    context - HoodieEngineContext
    
    writeConfig - instance of HoodieWriteConfig
    
    timelineService - Timeline Service that runs as part of write client.
- Method Detail
  - createIndex
```
protected abstract HoodieIndex<?,?> createIndex(HoodieWriteConfig writeConfig)
```
  - setOperationType
```
public void setOperationType(WriteOperationType operationType)
```
  - getOperationType
```
public WriteOperationType getOperationType()
```
  - getTableServiceClient
```
public BaseHoodieTableServiceClient<?,?,O> getTableServiceClient()
```
  - commit
```
public boolean commit(String instantTime,
                      O writeStatuses)
```
    Commit changes performed at the given instantTime marker.
  - commit
```
public boolean commit(String instantTime,
                      O writeStatuses,
                      Option<Map<String,String>> extraMetadata)
```
    Commit changes performed at the given instantTime marker.
  - commit
```
public boolean commit(String instantTime,
                      O writeStatuses,
                      Option<Map<String,String>> extraMetadata,
                      String commitActionType,
                      Map<String,List<String>> partitionToReplacedFileIds)
```
  - commit
```
public abstract boolean commit(String instantTime,
                               O writeStatuses,
                               Option<Map<String,String>> extraMetadata,
                               String commitActionType,
                               Map<String,List<String>> partitionToReplacedFileIds,
                               Option<BiConsumer<HoodieTableMetaClient,HoodieCommitMetadata>> extraPreCommitFunc)
```
  - commitStats
```
public boolean commitStats(String instantTime,
                           HoodieData<WriteStatus> writeStatuses,
                           List<HoodieWriteStat> stats,
                           Option<Map<String,String>> extraMetadata,
                           String commitActionType)
```
  - commitStats
```
public boolean commitStats(String instantTime,
                           HoodieData<WriteStatus> writeStatuses,
                           List<HoodieWriteStat> stats,
                           Option<Map<String,String>> extraMetadata,
                           String commitActionType,
                           Map<String,List<String>> partitionToReplaceFileIds,
                           Option<BiConsumer<HoodieTableMetaClient,HoodieCommitMetadata>> extraPreCommitFunc)
```
  - commit
```
protected void commit(HoodieTable table,
                      String commitActionType,
                      String instantTime,
                      HoodieCommitMetadata metadata,
                      List<HoodieWriteStat> stats,
                      HoodieData<WriteStatus> writeStatuses)
               throws IOException
```
    Throws:
    
    IOException
  - createTable
```
protected abstract HoodieTable<T,I,K,O> createTable(HoodieWriteConfig config,
                                                    org.apache.hadoop.conf.Configuration hadoopConf)
```
  - createTable
```
protected abstract HoodieTable<T,I,K,O> createTable(HoodieWriteConfig config,
                                                    org.apache.hadoop.conf.Configuration hadoopConf,
                                                    HoodieTableMetaClient metaClient)
```
  - preCommit
```
protected void preCommit(HoodieInstant inflightInstant,
                         HoodieCommitMetadata metadata)
```
    Any pre-commit actions like conflict resolution goes here.
    
    Parameters:
    
    inflightInstant - instant of inflight operation.
    
    metadata - commit metadata for which pre commit is being invoked.
  - writeTableMetadata
```
protected void writeTableMetadata(HoodieTable table,
                                  String instantTime,
                                  HoodieCommitMetadata metadata,
                                  HoodieData<WriteStatus> writeStatuses)
```
    Write the HoodieCommitMetadata to metadata table if available.
    
    Parameters:
    
    table - HoodieTable of interest.
    
    instantTime - instant time of the commit.
    
    metadata - instance of HoodieCommitMetadata.
    
    writeStatuses - WriteStatuses for the completed action.
  - filterExists
```
public abstract I filterExists(I hoodieRecords)
```
    Filter out HoodieRecords that already exists in the output folder. This is useful in deduplication.
    
    Parameters:
    
    hoodieRecords - Input Hoodie records.
    
    Returns:
    
    A subset of hoodieRecords, with existing records filtered out.
  - bootstrap
```
public void bootstrap(Option<Map<String,String>> extraMetadata)
```
    Main API to run bootstrap to hudi.
  - upsert
```
public abstract O upsert(I records,
                         String instantTime)
```
    Upsert a batch of new records into Hoodie table at the supplied instantTime.
    
    Parameters:
    
    records - hoodieRecords to upsert
    
    instantTime - Instant time of the commit
    
    Returns:
    
    WriteStatus to inspect errors and counts
  - upsertPreppedRecords
```
public abstract O upsertPreppedRecords(I preppedRecords,
                                       String instantTime)
```
    Upserts the given prepared records into the Hoodie table, at the supplied instantTime.
    This implementation requires that the input records are already tagged, and de-duped if needed.
    
    Parameters:
    
    preppedRecords - Prepared HoodieRecords to upsert
    
    instantTime - Instant time of the commit
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - insert
```
public abstract O insert(I records,
                         String instantTime)
```
    Inserts the given HoodieRecords, into the table. This API is intended to be used for normal writes.
    This implementation skips the index check and is able to leverage benefits such as small file handling/blocking alignment, as with upsert(), by profiling the workload
    
    Parameters:
    
    records - HoodieRecords to insert
    
    instantTime - Instant time of the commit
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - insertPreppedRecords
```
public abstract O insertPreppedRecords(I preppedRecords,
                                       String instantTime)
```
    Inserts the given prepared records into the Hoodie table, at the supplied instantTime.
    This implementation skips the index check, skips de-duping and is able to leverage benefits such as small file handling/blocking alignment, as with insert(), by profiling the workload. The prepared HoodieRecords should be de-duped if needed.
    
    Parameters:
    
    preppedRecords - HoodieRecords to insert
    
    instantTime - Instant time of the commit
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - bulkInsert
```
public abstract O bulkInsert(I records,
                             String instantTime)
```
    Loads the given HoodieRecords, as inserts into the table. This is suitable for doing big bulk loads into a Hoodie table for the very first time (e.g: converting an existing table to Hoodie).
    This implementation uses sortBy (which does range partitioning based on reservoir sampling) and attempts to control the numbers of files with less memory compared to the BaseHoodieWriteClient#insert(I, String)
    
    Parameters:
    
    records - HoodieRecords to insert
    
    instantTime - Instant time of the commit
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - bulkInsert
```
public abstract O bulkInsert(I records,
                             String instantTime,
                             Option<BulkInsertPartitioner> userDefinedBulkInsertPartitioner)
```
    Loads the given HoodieRecords, as inserts into the table. This is suitable for doing big bulk loads into a Hoodie table for the very first time (e.g: converting an existing table to Hoodie).
    This implementation uses sortBy (which does range partitioning based on reservoir sampling) and attempts to control the numbers of files with less memory compared to the BaseHoodieWriteClient#insert(I, String). Optionally it allows users to specify their own partitioner. If specified then it will be used for repartitioning records. See BulkInsertPartitioner.
    
    Parameters:
    
    records - HoodieRecords to insert
    
    instantTime - Instant time of the commit
    
    userDefinedBulkInsertPartitioner - If specified then it will be used to partition input records before they are inserted into hoodie.
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - bulkInsertPreppedRecords
```
public abstract O bulkInsertPreppedRecords(I preppedRecords,
                                           String instantTime,
                                           Option<BulkInsertPartitioner> bulkInsertPartitioner)
```
    Loads the given HoodieRecords, as inserts into the table. This is suitable for doing big bulk loads into a Hoodie table for the very first time (e.g: converting an existing table to Hoodie). The input records should contain no duplicates if needed.
    This implementation uses sortBy (which does range partitioning based on reservoir sampling) and attempts to control the numbers of files with less memory compared to the BaseHoodieWriteClient#insert(I, String). Optionally it allows users to specify their own partitioner. If specified then it will be used for repartitioning records. See BulkInsertPartitioner.
    
    Parameters:
    
    preppedRecords - HoodieRecords to insert
    
    instantTime - Instant time of the commit
    
    bulkInsertPartitioner - If specified then it will be used to partition input records before they are inserted into hoodie.
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - delete
```
public abstract O delete(K keys,
                         String instantTime)
```
    Deletes a list of HoodieKeys from the Hoodie table, at the supplied instantTime HoodieKeys will be de-duped and non-existent keys will be removed before deleting.
    
    Parameters:
    
    keys - List of HoodieKeys to be deleted
    
    instantTime - Commit time handle
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - deletePrepped
```
public abstract O deletePrepped(I preppedRecords,
                                String instantTime)
```
    Delete records from Hoodie table based on HoodieKey and HoodieRecordLocation specified in preppedRecords.
    
    Parameters:
    
    preppedRecords - Empty records with key and locator set.
    
    instantTime - Commit time handle.
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts.
  - preWrite
```
public void preWrite(String instantTime,
                     WriteOperationType writeOperationType,
                     HoodieTableMetaClient metaClient)
```
    Common method containing steps to be performed before write (upsert/insert/...
    
    Parameters:
    
    instantTime -
    
    writeOperationType -
    
    metaClient -
  - postWrite
```
public O postWrite(HoodieWriteMetadata<O> result,
                   String instantTime,
                   HoodieTable hoodieTable)
```
    Common method containing steps to be performed after write (upsert/insert/..) operations including auto-commit.
    
    Parameters:
    
    result - Commit Action Result
    
    instantTime - Instant Time
    
    hoodieTable - Hoodie Table
    
    Returns:
    
    Write Status
  - postCommit
```
protected void postCommit(HoodieTable table,
                          HoodieCommitMetadata metadata,
                          String instantTime,
                          Option<Map<String,String>> extraMetadata)
```
    Post Commit Hook. Derived classes use this method to perform post-commit processing
    
    Parameters:
    
    table - table to commit on
    
    metadata - Commit Metadata corresponding to committed instant
    
    instantTime - Instant Time
    
    extraMetadata - Additional Metadata passed by user
  - mayBeCleanAndArchive
```
protected void mayBeCleanAndArchive(HoodieTable table)
```
    Triggers cleaning and archival for the table of interest. This method is called outside of locks. So, internal callers should ensure they acquire lock whereever applicable.
    
    Parameters:
    
    table - instance of HoodieTable of interest.
  - runTableServicesInline
```
protected void runTableServicesInline(HoodieTable table,
                                      HoodieCommitMetadata metadata,
                                      Option<Map<String,String>> extraMetadata)
```
  - autoCleanOnCommit
```
protected void autoCleanOnCommit()
```
  - autoArchiveOnCommit
```
protected void autoArchiveOnCommit(HoodieTable table)
```
  - runAnyPendingCompactions
```
public void runAnyPendingCompactions()
```
    Run any pending compactions.
  - runAnyPendingLogCompactions
```
public void runAnyPendingLogCompactions()
```
    Run any pending log compactions.
  - savepoint
```
public void savepoint(String user,
                      String comment)
```
    Create a savepoint based on the latest commit action on the timeline.
    
    Parameters:
    
    user - User creating the savepoint
    
    comment - Comment for the savepoint
  - savepoint
```
public void savepoint(String instantTime,
                      String user,
                      String comment)
```
    Savepoint a specific commit instant time. Latest version of data files as of the passed in instantTime will be referenced in the savepoint and will never be cleaned. The savepointed commit will never be rolledback or archived.
    This gives an option to rollback the state to the savepoint anytime. Savepoint needs to be manually created and deleted.
    Savepoint should be on a commit that could not have been cleaned.
    
    Parameters:
    
    instantTime - Commit that should be savepointed
    
    user - User creating the savepoint
    
    comment - Comment for the savepoint
  - deleteSavepoint
```
public void deleteSavepoint()
```
    Delete a savepoint based on the latest commit action on the savepoint timeline.
  - deleteSavepoint
```
public void deleteSavepoint(String savepointTime)
```
    Delete a savepoint that was created. Once the savepoint is deleted, the commit can be rolledback and cleaner may clean up data files.
    
    Parameters:
    
    savepointTime - Savepoint time to delete
  - restoreToSavepoint
```
public void restoreToSavepoint()
```
    Restore the data to a savepoint based on the latest commit action on the savepoint timeline.
  - restoreToSavepoint
```
public void restoreToSavepoint(String savepointTime)
```
    Restore the data to the savepoint. WARNING: This rolls back recent commits and deleted data files and also pending compactions after savepoint time. Queries accessing the files will mostly fail. This is expected to be a manual operation and no concurrent write or compaction is expected to be running
    
    Parameters:
    
    savepointTime - Savepoint time to rollback to
  - rollback
```
@Deprecated
public boolean rollback(String commitInstantTime)
                             throws HoodieRollbackException
```
    Deprecated.
    
    Throws:
    
    HoodieRollbackException
  - rollback
```
@Deprecated
public boolean rollback(String commitInstantTime,
                                    String rollbackInstantTimestamp)
                             throws HoodieRollbackException
```
    Deprecated.
    
    Throws:
    
    HoodieRollbackException
  - restoreToInstant
```
public HoodieRestoreMetadata restoreToInstant(String savepointToRestoreTimestamp,
                                              boolean initialMetadataTableIfNecessary)
                                       throws HoodieRestoreException
```
    NOTE : This action requires all writers (ingest and compact) to a table to be stopped before proceeding. Revert the (inflight/committed) record changes for all commits after the provided instant time.
    
    Parameters:
    
    savepointToRestoreTimestamp - savepoint instant time to which restoration is requested
    
    Throws:
    
    HoodieRestoreException
  - clean
```
public HoodieCleanMetadata clean(String cleanInstantTime)
                          throws HoodieIOException
```
    Clean up any stale/old files/data lying around (either on file storage or index storage) based on the configurations and CleaningPolicy used. (typically files that no longer can be used by a running query can be cleaned)
    
    Throws:
    
    HoodieIOException
  - clean
```
@Deprecated
public HoodieCleanMetadata clean(String cleanInstantTime,
                                             boolean skipLocking)
                                      throws HoodieIOException
```
    Deprecated.
    
    Clean up any stale/old files/data lying around (either on file storage or index storage) based on the configurations and CleaningPolicy used. (typically files that no longer can be used by a running query can be cleaned)
    
    Parameters:
    
    cleanInstantTime - instant time for clean.
    
    skipLocking - if this is triggered by another parent transaction, locking can be skipped.
    
    Returns:
    
    instance of HoodieCleanMetadata.
    
    Throws:
    
    HoodieIOException
  - clean
```
public HoodieCleanMetadata clean(String cleanInstantTime,
                                 boolean scheduleInline,
                                 boolean skipLocking)
                          throws HoodieIOException
```
    Clean up any stale/old files/data lying around (either on file storage or index storage) based on the configurations and CleaningPolicy used. (typically files that no longer can be used by a running query can be cleaned). This API provides the flexibility to schedule clean instant asynchronously via scheduleTableService(String, Option, TableServiceType) and disable inline scheduling of clean.
    
    Parameters:
    
    cleanInstantTime - instant time for clean.
    
    scheduleInline - true if needs to be scheduled inline. false otherwise.
    
    skipLocking - if this is triggered by another parent transaction, locking can be skipped.
    
    Throws:
    
    HoodieIOException
  - clean
```
public HoodieCleanMetadata clean()
```
  - clean
```
@Deprecated
public HoodieCleanMetadata clean(boolean skipLocking)
```
    Deprecated.
    
    Triggers clean for the table. This refers to Clean up any stale/old files/data lying around (either on file storage or index storage) based on the * configurations and CleaningPolicy used.
    
    Parameters:
    
    skipLocking - if this is triggered by another parent transaction, locking can be skipped.
    
    Returns:
    
    instance of HoodieCleanMetadata.
  - archive
```
protected void archive(HoodieTable table)
```
    Trigger archival for the table. This ensures that the number of commits do not explode and keep increasing unbounded over time.
    
    Parameters:
    
    table - table to commit on.
  - archive
```
public void archive()
```
    Trigger archival for the table. This ensures that the number of commits do not explode and keep increasing unbounded over time.
  - startCommit
```
public String startCommit()
```
    Provides a new commit time for a write operation (insert/update/delete).
  - startCommit
```
public String startCommit(String actionType,
                          HoodieTableMetaClient metaClient)
```
    Provides a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) with specified action.
  - startCommitWithTime
```
public void startCommitWithTime(String instantTime)
```
    Provides a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) without specified action.
    
    Parameters:
    
    instantTime - Instant time to be generated
  - startCommitWithTime
```
public void startCommitWithTime(String instantTime,
                                String actionType)
```
    Completes a new commit time for a write operation (insert/update/delete/insert_overwrite/insert_overwrite_table) with specified action.
  - scheduleCompaction
```
public Option<String> scheduleCompaction(Option<Map<String,String>> extraMetadata)
                                  throws HoodieIOException
```
    Schedules a new compaction instant.
    
    Parameters:
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - scheduleCompactionAtInstant
```
public boolean scheduleCompactionAtInstant(String instantTime,
                                           Option<Map<String,String>> extraMetadata)
                                    throws HoodieIOException
```
    Schedules a new compaction instant with passed-in instant time.
    
    Parameters:
    
    instantTime - Compaction Instant Time
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - scheduleIndexing
```
public Option<String> scheduleIndexing(List<MetadataPartitionType> partitionTypes)
```
    Schedules INDEX action.
    
    Parameters:
    
    partitionTypes - - list of MetadataPartitionType which needs to be indexed
    
    Returns:
    
    instant time for the requested INDEX action
  - index
```
public Option<HoodieIndexCommitMetadata> index(String indexInstantTime)
```
    Runs INDEX action to build out the metadata partitions as planned for the given instant time.
    
    Parameters:
    
    indexInstantTime - - instant time for the requested INDEX action
    
    Returns:
    
    Option after successful indexing.
  - dropIndex
```
public void dropIndex(List<MetadataPartitionType> partitionTypes)
```
    Drops the index and removes the metadata partitions.
    
    Parameters:
    
    partitionTypes - - list of MetadataPartitionType which needs to be indexed
  - cluster
```
public HoodieWriteMetadata<O> cluster(String clusteringInstantTime)
```
    Performs Clustering for the workload stored in instant-time.
    
    Parameters:
    
    clusteringInstantTime - Clustering Instant Time
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - compact
```
public HoodieWriteMetadata<O> compact(String compactionInstantTime)
```
    Performs Compaction for the workload stored in instant-time.
    
    Parameters:
    
    compactionInstantTime - Compaction Instant Time
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - commitCompaction
```
public void commitCompaction(String compactionInstantTime,
                             HoodieCommitMetadata metadata,
                             Option<Map<String,String>> extraMetadata)
```
    Commit a compaction operation. Allow passing additional meta-data to be stored in commit instant file.
    
    Parameters:
    
    compactionInstantTime - Compaction Instant Time
    
    metadata - All the metadata that gets stored along with a commit
    
    extraMetadata - Extra Metadata to be stored
  - completeCompaction
```
protected void completeCompaction(HoodieCommitMetadata metadata,
                                  HoodieTable table,
                                  String compactionCommitTime)
```
    Commit Compaction and track metrics.
  - scheduleLogCompaction
```
public Option<String> scheduleLogCompaction(Option<Map<String,String>> extraMetadata)
                                     throws HoodieIOException
```
    Schedules a new log compaction instant.
    
    Parameters:
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - scheduleLogCompactionAtInstant
```
public boolean scheduleLogCompactionAtInstant(String instantTime,
                                              Option<Map<String,String>> extraMetadata)
                                       throws HoodieIOException
```
    Schedules a new log compaction instant with passed-in instant time.
    
    Parameters:
    
    instantTime - Log Compaction Instant Time
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - logCompact
```
public HoodieWriteMetadata<O> logCompact(String logCompactionInstantTime)
```
    Performs Log Compaction for the workload stored in instant-time.
    
    Parameters:
    
    logCompactionInstantTime - Log Compaction Instant Time
    
    Returns:
    
    Collection of WriteStatus to inspect errors and counts
  - commitLogCompaction
```
public void commitLogCompaction(String logCompactionInstantTime,
                                HoodieCommitMetadata metadata,
                                Option<Map<String,String>> extraMetadata)
```
    Commit a log compaction operation. Allow passing additional meta-data to be stored in commit instant file.
    
    Parameters:
    
    logCompactionInstantTime - Log Compaction Instant Time
    
    metadata - All the metadata that gets stored along with a commit
    
    extraMetadata - Extra Metadata to be stored
  - completeLogCompaction
```
protected void completeLogCompaction(HoodieCommitMetadata metadata,
                                     HoodieTable table,
                                     String logCompactionCommitTime)
```
    Commit Log Compaction and track metrics.
  - compact
```
protected HoodieWriteMetadata<O> compact(String compactionInstantTime,
                                         boolean shouldComplete)
```
    Ensures compaction instant is in expected state and performs Compaction for the workload stored in instant-time.
    
    Parameters:
    
    compactionInstantTime - Compaction Instant Time
    
    Returns:
    
    Collection of Write Status
  - inlineScheduleCompaction
```
protected Option<String> inlineScheduleCompaction(Option<Map<String,String>> extraMetadata)
```
    Schedules compaction inline.
    
    Parameters:
    
    extraMetadata - extra metadata to be used.
    
    Returns:
    
    compaction instant if scheduled.
  - logCompact
```
protected HoodieWriteMetadata<O> logCompact(String logCompactionInstantTime,
                                            boolean shouldComplete)
```
    Ensures compaction instant is in expected state and performs Log Compaction for the workload stored in instant-time.s
    
    Parameters:
    
    logCompactionInstantTime - Compaction Instant Time
    
    Returns:
    
    Collection of Write Status
  - scheduleClustering
```
public Option<String> scheduleClustering(Option<Map<String,String>> extraMetadata)
                                  throws HoodieIOException
```
    Schedules a new clustering instant.
    
    Parameters:
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - scheduleClusteringAtInstant
```
public boolean scheduleClusteringAtInstant(String instantTime,
                                           Option<Map<String,String>> extraMetadata)
                                    throws HoodieIOException
```
    Schedules a new clustering instant with passed-in instant time.
    
    Parameters:
    
    instantTime - clustering Instant Time
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - scheduleCleaningAtInstant
```
protected boolean scheduleCleaningAtInstant(String instantTime,
                                            Option<Map<String,String>> extraMetadata)
                                     throws HoodieIOException
```
    Schedules a new cleaning instant with passed-in instant time.
    
    Parameters:
    
    instantTime - cleaning Instant Time
    
    extraMetadata - Extra Metadata to be stored
    
    Throws:
    
    HoodieIOException
  - cluster
```
public HoodieWriteMetadata<O> cluster(String clusteringInstant,
                                      boolean shouldComplete)
```
    Ensures clustering instant is in expected state and performs clustering for the plan stored in metadata.
    
    Parameters:
    
    clusteringInstant - Clustering Instant Time
    
    Returns:
    
    Collection of Write Status
  - scheduleTableService
```
public Option<String> scheduleTableService(Option<Map<String,String>> extraMetadata,
                                           TableServiceType tableServiceType)
```
    Schedule table services such as clustering, compaction & cleaning.
    
    Parameters:
    
    extraMetadata - Metadata to pass onto the scheduled service instant
    
    tableServiceType - Type of table service to schedule
    
    Returns:
    
    The given instant time option or empty if no table service plan is scheduled
  - scheduleTableService
```
public Option<String> scheduleTableService(String instantTime,
                                           Option<Map<String,String>> extraMetadata,
                                           TableServiceType tableServiceType)
```
    Schedule table services such as clustering, compaction & cleaning.
    
    Parameters:
    
    extraMetadata - Metadata to pass onto the scheduled service instant
    
    tableServiceType - Type of table service to schedule
    
    Returns:
    
    The given instant time option or empty if no table service plan is scheduled
  - getMetrics
```
public HoodieMetrics getMetrics()
```
  - getIndex
```
public HoodieIndex<?,?> getIndex()
```
  - doInitTable
```
protected void doInitTable(WriteOperationType operationType,
                           HoodieTableMetaClient metaClient,
                           Option<String> instantTime)
```
    Performs necessary bootstrapping operations (for ex, validating whether Metadata Table has to be bootstrapped).
    NOTE: THIS OPERATION IS EXECUTED UNDER LOCK, THEREFORE SHOULD AVOID ANY OPERATIONS NOT REQUIRING EXTERNAL SYNCHRONIZATION
    
    Parameters:
    
    metaClient - instance of HoodieTableMetaClient
    
    instantTime - current inflight instant time
  - initMetadataTable
```
protected void initMetadataTable(Option<String> instantTime)
```
    Bootstrap the metadata table.
    
    Parameters:
    
    instantTime - current inflight instant time
  - initTable
```
protected final HoodieTable initTable(WriteOperationType operationType,
                                      Option<String> instantTime,
                                      boolean initMetadataTable)
```
  - initTable
```
public final HoodieTable initTable(WriteOperationType operationType,
                                   Option<String> instantTime)
```
    Instantiates and initializes instance of HoodieTable, performing crucial bootstrapping operations such as: NOTE: This method is engine-agnostic and SHOULD NOT be overloaded, please check on doInitTable(WriteOperationType, HoodieTableMetaClient, Option) instead
    - Checking whether upgrade/downgrade is required
    - Bootstrapping Metadata Table (if required)
    - Initializing metrics contexts
  - setWriteSchemaForDeletes
```
protected void setWriteSchemaForDeletes(HoodieTableMetaClient metaClient)
```
    Sets write schema from last instant since deletes may not have schema set in the config.
  - releaseResources
```
protected void releaseResources(String instantTime)
```
    Called after each write, to release any resources used.
  - close
```
public void close()
```
    Description copied from class: BaseHoodieClient
    
    Releases any resources used by the client.
    
    Specified by:
    
    close in interface AutoCloseable
    
    Overrides:
    
    close in class BaseHoodieClient
  - tryUpgrade
```
protected void tryUpgrade(HoodieTableMetaClient metaClient,
                          Option<String> instantTime)
```
    Upgrades the hoodie table if need be when moving to a new Hudi version. This method is called within a lock. Try to avoid double locking from within this method.
    
    Parameters:
    
    metaClient - instance of HoodieTableMetaClient to use.
    
    instantTime - instant time of interest if we have one.
  - lazyRollbackFailedIndexing
```
public boolean lazyRollbackFailedIndexing()
```
    Rolls back the failed delta commits corresponding to the indexing action.
    TODO(HUDI-5733): This should be cleaned up once the proper fix of rollbacks in the metadata table is landed.
    
    Returns:
    
    true if rollback happens; false otherwise.
  - rollbackFailedWrites
```
public boolean rollbackFailedWrites()
```
    Rollback failed writes if any.
    
    Returns:
    
    true if rollback happened. false otherwise.
  - addColumn
```
public void addColumn(String colName,
                      org.apache.avro.Schema schema,
                      String doc,
                      String position,
                      TableChange.ColumnPositionChange.ColumnPositionType positionType)
```
    add columns to table.
    
    Parameters:
    
    colName - col name to be added. if we want to add col to a nested filed, the fullName should be specified
    
    schema - col type to be added.
    
    doc - col doc to be added.
    
    position - col position to be added
    
    positionType - col position change type. now support three change types: first/after/before
  - addColumn
```
public void addColumn(String colName,
                      org.apache.avro.Schema schema)
```
  - deleteColumns
```
public void deleteColumns(String... colNames)
```
    delete columns to table.
    
    Parameters:
    
    colNames - col name to be deleted. if we want to delete col from a nested filed, the fullName should be specified
  - renameColumn
```
public void renameColumn(String colName,
                         String newName)
```
    rename col name for hudi table.
    
    Parameters:
    
    colName - col name to be renamed. if we want to rename col from a nested filed, the fullName should be specified
    
    newName - new name for current col. no need to specify fullName.
  - updateColumnNullability
```
public void updateColumnNullability(String colName,
                                    boolean nullable)
```
    update col nullable attribute for hudi table.
    
    Parameters:
    
    colName - col name to be changed. if we want to change col from a nested filed, the fullName should be specified
    
    nullable - .
  - updateColumnType
```
public void updateColumnType(String colName,
                             Type newType)
```
    update col Type for hudi table. only support update primitive type to primitive type. cannot update nest type to nest type or primitive type eg: RecordType -> MapType, MapType -> LongType.
    
    Parameters:
    
    colName - col name to be changed. if we want to change col from a nested filed, the fullName should be specified
    
    newType - .
  - updateColumnComment
```
public void updateColumnComment(String colName,
                                String doc)
```
    update col comment for hudi table.
    
    Parameters:
    
    colName - col name to be changed. if we want to change col from a nested filed, the fullName should be specified
    
    doc - .
  - reOrderColPosition
```
public void reOrderColPosition(String colName,
                               String referColName,
                               TableChange.ColumnPositionChange.ColumnPositionType orderType)
```
    reorder the position of col.
    
    Parameters:
    
    colName - column which need to be reordered. if we want to change col from a nested filed, the fullName should be specified.
    
    referColName - reference position.
    
    orderType - col position change type. now support three change types: first/after/before

Class BaseHoodieWriteClient<T,I,K,O>

Field Summary

Fields inherited from class org.apache.hudi.client.BaseHoodieClient

Constructor Summary

Method Summary

Methods inherited from class org.apache.hudi.client.BaseHoodieClient

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.hudi.client.RunsTableService

Field Detail

LOOKUP_STR

writeTimer

lastCompletedTxnAndMetadata

pendingInflightAndRequestedInstants

tableServiceClient

Constructor Detail

BaseHoodieWriteClient

BaseHoodieWriteClient

Method Detail

createIndex

setOperationType

getOperationType

getTableServiceClient

commit

commit

commit

commit

commitStats

commitStats

commit

createTable

createTable

preCommit

writeTableMetadata

filterExists

bootstrap

upsert

upsertPreppedRecords

insert

insertPreppedRecords

bulkInsert

bulkInsert

bulkInsertPreppedRecords

delete

deletePrepped

preWrite

postWrite

postCommit

mayBeCleanAndArchive

runTableServicesInline

autoCleanOnCommit

autoArchiveOnCommit

runAnyPendingCompactions

runAnyPendingLogCompactions

savepoint

savepoint

deleteSavepoint

deleteSavepoint

restoreToSavepoint

restoreToSavepoint

rollback

rollback

restoreToInstant

clean

clean

clean

clean

clean

archive

archive

startCommit

startCommit

startCommitWithTime

startCommitWithTime

scheduleCompaction

scheduleCompactionAtInstant

scheduleIndexing

index

dropIndex

cluster

compact