case class DeltaCommitFileProvider(logPath: String, maxVersionInclusive: Long, uuids: Map[Long, String]) extends Product with Serializable

Provides access to resolve Delta commit files names based on the commit-version.

This class is part of the changes introduced to accommodate the adoption of coordinated-commits in Delta Lake. Previously, certain code paths assumed the existence of delta files for a specific version at a predictable path _delta_log/$version.json. With coordinated-commits, delta files may alternatively be located at _delta_log/_commits/$version.$uuid.json. DeltaCommitFileProvider attempts to locate the correct delta files from the Snapshot's LogSegment.

logPath

The path to the Delta table log directory.

maxVersionInclusive

The maximum version of the Delta table (inclusive).

uuids

A map of version numbers to their corresponding UUIDs.

Linear Supertypes
Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaCommitFileProvider
  2. Serializable
  3. Product
  4. Equals
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new DeltaCommitFileProvider(logPath: String, maxVersionInclusive: Long, uuids: Map[Long, String])

    logPath

    The path to the Delta table log directory.

    maxVersionInclusive

    The maximum version of the Delta table (inclusive).

    uuids

    A map of version numbers to their corresponding UUIDs.

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. def deltaFile(version: Long): Path
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  9. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  11. def listSortedUnbackfilledDeltaFiles(startVersionOpt: Option[Long] = None): Seq[(Long, Path)]

    Lists unbackfilled delta files in a sorted order without incurring additional IO operations.

  12. val logPath: String
  13. val maxVersionInclusive: Long
  14. lazy val minUnbackfilledVersion: Long
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. def productElementNames: Iterator[String]
    Definition Classes
    Product
  19. lazy val resolvedPath: Path
    Annotations
    @transient()
  20. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  21. val uuids: Map[Long, String]
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped