Type Members

1.  trait AbstractBatchBackfillingCommitOwnerClient extends CommitOwnerClient with Logging

   An abstract CommitOwnerClient which triggers backfills every n commits.

   An abstract CommitOwnerClient which triggers backfills every n commits. - every commit version which satisfies commitVersion % batchSize == 0 will trigger a backfill.

2.  case class Commit(version: Long, fileStatus: FileStatus, commitTimestamp: Long) extends Product with Serializable

   Representation of a commit file

3.  case class CommitFailedException(retryable: Boolean, conflict: Boolean, message: String) extends Exception with Product with Serializable

   Exception raised by CommitOwnerClient.commit method.

   Exception raised by CommitOwnerClient.commit method. | retryable | conflict | meaning | | no | no | something bad happened (e.g. auth failure) | | no | yes | permanent transaction conflict (e.g. multi-table commit failed) | | yes | no | transient error (e.g. network hiccup) | | yes | yes | physical conflict (allowed to rebase and retry) |

4.  trait CommitOwnerBuilder extends AnyRef

   A builder interface for CommitOwnerClient

5.  trait CommitOwnerClient extends AnyRef

   CommitOwnerClient is responsible for managing commits for a managed-commit delta table.

   CommitOwnerClient is responsible for managing commits for a managed-commit delta table. 1. It provides API to commit a new version of the table. See CommitOwnerClient.commit API. 2. It makes sure that commits are backfilled if/when needed 3. It also tracks and returns unbackfilled commits. See CommitOwnerClient.getCommits API.

6.  case class CommitResponse(commit: Commit) extends Product with Serializable

   Response container for CommitOwnerClient.commit API

7.  case class GetCommitsResponse(commits: Seq[Commit], latestTableVersion: Long) extends Product with Serializable

   Response container for CommitOwnerClient.getCommits API

8.  class InMemoryCommitOwner extends AbstractBatchBackfillingCommitOwnerClient
9.  case class InMemoryCommitOwnerBuilder(batchSize: Long) extends CommitOwnerBuilder with Product with Serializable

   The InMemoryCommitOwnerBuilder class is responsible for creating singleton instances of InMemoryCommitOwner with the specified batchSize.

10.  case class TableCommitOwnerClient(commitOwnerClient: CommitOwnerClient, logPath: Path, tableConf: Map[String, String], hadoopConf: Configuration, logStore: LogStore) extends Product with Serializable

    A wrapper around CommitOwnerClient that provides a more user-friendly API for committing / accessing commits to a specific table.

    A wrapper around CommitOwnerClient that provides a more user-friendly API for committing / accessing commits to a specific table. This class takes care of passing the table specific configuration to the underlying CommitOwnerClient e.g. logPath / logStore / managedCommitTableConf / hadoopConf.

    commitOwnerClient

    the underlying CommitOwnerClient

    logPath

    the path to the log directory

    tableConf

    the table specific managed-commit configuration

    hadoopConf

    hadoop configuration

    logStore

    the log store

11.  case class UpdatedActions(commitInfo: CommitInfo, newMetadata: Metadata, newProtocol: Protocol, oldMetadata: Metadata, oldProtocol: Protocol) extends Product with Serializable

    A container class to inform the CommitOwnerClient about any changes in Protocol/Metadata