Packages

c

org.apache.spark.sql.delta

UninitializedV2CheckpointProvider

case class UninitializedV2CheckpointProvider(version: Long, fileStatus: FileStatus, logPath: Path, hadoopConf: Configuration, deltaLogOptions: Map[String, String], logStore: LogStore, lastCheckpointInfoOpt: Option[LastCheckpointInfo]) extends UninitializedV2LikeCheckpointProvider with Product with Serializable

An implementation of UninitializedCheckpointProvider to for v2 checkpoints. This needs to be resolved into a V2CheckpointProvider. This class starts an I/O to fetch the V2 actions (CheckpointMetadata, SidecarFile) as soon as the class is initialized so that the extra overhead could be parallelized with other operations like reading CRC.

Linear Supertypes
Serializable, Product, Equals, UninitializedV2LikeCheckpointProvider, UninitializedCheckpointProvider, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. UninitializedV2CheckpointProvider
  2. Serializable
  3. Product
  4. Equals
  5. UninitializedV2LikeCheckpointProvider
  6. UninitializedCheckpointProvider
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new UninitializedV2CheckpointProvider(version: Long, fileStatus: FileStatus, logPath: Path, hadoopConf: Configuration, deltaLogOptions: Map[String, String], logStore: LogStore, lastCheckpointInfoOpt: Option[LastCheckpointInfo])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. val deltaLogOptions: Map[String, String]
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. val fileStatus: FileStatus
    Definition Classes
    UninitializedV2CheckpointProvider → UninitializedV2LikeCheckpointProvider
  9. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  10. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. val hadoopConf: Configuration
  12. def isEmpty: Boolean

    True if the checkpoint provider is empty (does not refer to a valid checkpoint)

    True if the checkpoint provider is empty (does not refer to a valid checkpoint)

    Definition Classes
    UninitializedCheckpointProvider
  13. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  14. val lastCheckpointInfoOpt: Option[LastCheckpointInfo]
    Definition Classes
    UninitializedV2CheckpointProvider → UninitializedV2LikeCheckpointProvider
  15. val logPath: Path
    Definition Classes
    UninitializedV2CheckpointProvider → UninitializedV2LikeCheckpointProvider
  16. val logStore: LogStore
  17. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. val nonFateSharingCheckpointReadFuture: NonFateSharingFuture[(Option[CheckpointMetadata], Seq[SidecarFile])]
  19. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. def productElementNames: Iterator[String]
    Definition Classes
    Product
  22. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  23. lazy val topLevelFileIndex: Option[DeltaLogFileIndex]

    File index which could help derive actions stored in top level files for the checkpoint.

    File index which could help derive actions stored in top level files for the checkpoint. This could be used to get Protocol, Metadata etc from a checkpoint. This could also be used if we want to shallow copy a checkpoint.

    Definition Classes
    UninitializedV2LikeCheckpointProvider → UninitializedCheckpointProvider
  24. lazy val topLevelFiles: Seq[FileStatus]

    Top level files that represents this checkpoint.

    Top level files that represents this checkpoint. These files could be reused again to initialize the CheckpointProvider.

    Definition Classes
    UninitializedV2LikeCheckpointProvider → UninitializedCheckpointProvider
  25. val v2CheckpointFormat: Format
    Definition Classes
    UninitializedV2CheckpointProvider → UninitializedV2LikeCheckpointProvider
  26. val version: Long

    Checkpoint version

    Checkpoint version

    Definition Classes
    UninitializedV2CheckpointProvider → UninitializedCheckpointProvider
  27. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  28. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  29. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from UninitializedV2LikeCheckpointProvider

Inherited from UninitializedCheckpointProvider

Inherited from AnyRef

Inherited from Any

Ungrouped