Packages

c

io.delta.flink.sink.internal.writer

DeltaWriterBucketState

class DeltaWriterBucketState extends AnyRef

State of a DeltaWriterBucket that will become part of each application's snapshot created during pre-commit phase of a checkpoint process or manually on demand by the user. see Fault Tolerance via State Snapshots section on

See also

this page This class is partially inspired by org.apache.flink.connector.file.sink.writer.FileWriterBucketState but with some modifications like:

  • removed snapshotting in-progress file's state because io.delta.flink.sink.DeltaSink is supposed to always roll part files on checkpoint so there is no need to recover any in-progress files' states
  • extends the state by adding application's unique identifier to guarantee the idempotent file writes and commits to the io.delta.standalone.DeltaLog Lifecycle of instances of this class is as follows:
  • Every instance is being created via DeltaWriter#snapshotState() method at the finish phase of each checkpoint interval and serialized as a part of snapshotted app's state.
  • It can be also created by the Flink framework itself during failure/snapshot recovery when it's deserialized from the snapshotted state and provided as input param collection to io.delta.flink.sink.DeltaSink#createWriter
Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaWriterBucketState
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DeltaWriterBucketState(bucketId: String, bucketPath: Path, appId: String)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. def getAppId(): String
  10. def getBucketId(): String
  11. def getBucketPath(): Path
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  16. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  17. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  19. def toString(): String
    Definition Classes
    DeltaWriterBucketState → AnyRef → Any
    Annotations
    @Override()
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped