Packages

c

io.delta.flink.source.internal.state

DeltaEnumeratorStateCheckpoint

class DeltaEnumeratorStateCheckpoint[SplitT <: DeltaSourceSplit] extends AnyRef

A checkpoint of the current state of SplitEnumerator.

It contains all necessary information need by SplitEnumerator to resume work after checkpoint recovery including currently pending splits that are not yet assigned and resume changes discovery task on Delta table in Boundedness#CONTINUOUS_UNBOUNDED mode

During checkpoint, Flink will serialize this object and persist it in checkpoint location. During the recovery, Flink will deserialize this object from Checkpoint/Savepoint and will use it to recreate SplitEnumerator.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaEnumeratorStateCheckpoint
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DeltaEnumeratorStateCheckpoint(deltaTablePath: Path, snapshotVersion: Long, monitoringForChanges: Boolean, pendingSplitsCheckpoint: PendingSplitsCheckpoint[SplitT])

    Attributes
    protected[state]

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. def getAlreadyProcessedPaths(): Collection[Path]

    returns

    The paths that are no longer in the enumerator checkpoint, but have been processed before and should be ignored.

  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def getDeltaTablePath(): Path

    returns

    Path to Delta Table used for this snapshot.

  12. def getSnapshotVersion(): Long

    returns

    The initial version of Delta Table from witch we started reading the Delta Table.

  13. def getSplits(): Collection[SplitT]

    returns

    The checkpointed DeltaSourceSplit that were not yet assigned to file readers.

  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. def isMonitoringForChanges(): Boolean

    returns

    Boolean flag indicating that DeltaSourceSplitEnumerator started monitoring for changes on Delta Table.

  17. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  18. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  21. def toString(): String
    Definition Classes
    AnyRef → Any
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped