Packages

class ForeachBatchSink[T] extends Sink

Linear Supertypes
Sink, Table, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ForeachBatchSink
  2. Sink
  3. Table
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ForeachBatchSink(batchWriter: (Dataset[T], Long) ⇒ Unit, encoder: ExpressionEncoder[T])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def addBatch(batchId: Long, data: DataFrame): Unit

    Adds a batch of data to this sink.

    Adds a batch of data to this sink. The data for a given batchId is deterministic and if this method is called more than once with the same batchId (which will happen in the case of failures), then data should only be added once.

    Note 1: You cannot apply any operators on data except consuming it (e.g., collect/foreach). Otherwise, you may get a wrong result.

    Note 2: The method is supposed to be executed synchronously, i.e. the method should only return after data is consumed by sink successfully.

    Definition Classes
    ForeachBatchSink → Sink
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def capabilities(): Set[TableCapability]
    Definition Classes
    Sink → Table
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  8. def columns(): Array[connector.catalog.Column]
    Definition Classes
    Table
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  15. def name(): String
    Definition Classes
    Sink → Table
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. def partitioning(): Array[Transform]
    Definition Classes
    Table
  20. def properties(): Map[String, String]
    Definition Classes
    Table
  21. def schema(): StructType
    Definition Classes
    Sink → Table
  22. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  23. def toString(): String
    Definition Classes
    ForeachBatchSink → AnyRef → Any
  24. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Sink

Inherited from Table

Inherited from AnyRef

Inherited from Any

Ungrouped