Packages

package files

Type Members

  1. class DelayedCommitProtocol extends FileCommitProtocol with Serializable with Logging

    Writes out the files to path and returns a list of them in addedStatuses.

  2. class DeltaSourceSnapshot extends SnapshotIterator with StateCache

    Converts a Snapshot into the initial set of files read when starting a new streaming query.

    Converts a Snapshot into the initial set of files read when starting a new streaming query. The list of files that represent the table at the time the query starts are selected by: - Adding version and index to each file to enable splitting of the initial state into multiple batches. - Filtering files that don't match partition predicates, while preserving the aforementioned indexing.

  3. trait SQLMetricsReporting extends AnyRef

    This trait is used to register SQL metrics for a Delta Operation.

    This trait is used to register SQL metrics for a Delta Operation. Registering will allow the metrics to be instrumented via the CommitInfo and is accessible via DescribeHistory

  4. trait SnapshotIterator extends AnyRef
  5. class TahoeBatchFileIndex extends TahoeFileIndex

    A TahoeFileIndex that generates the list of files from a given list of files that are within a version range of DeltaLog.

  6. abstract class TahoeFileIndex extends FileIndex

    A FileIndex that generates the list of files managed by the Tahoe protocol.

  7. case class TahoeLogFileIndex(spark: SparkSession, deltaLog: DeltaLog, path: Path, partitionFilters: Seq[Expression] = Nil, versionToUse: Option[Long] = None) extends TahoeFileIndex with Product with Serializable

    A TahoeFileIndex that generates the list of files from DeltaLog with given partition filters.

  8. trait TransactionalWrite extends DeltaLogging

    Adds the ability to write files out as part of a transaction.

    Adds the ability to write files out as part of a transaction. Checks are performed to ensure that the data being written matches either the current metadata or the new metadata being set by this transaction.

Ungrouped