Packages

c

io.delta.flink.source

RowDataBoundedDeltaSourceBuilder

class RowDataBoundedDeltaSourceBuilder extends BoundedDeltaSourceBuilder[RowData, RowDataBoundedDeltaSourceBuilder]

A builder class for DeltaSource for a stream of RowData where the created source instance will operate in Bounded mode.

For most common use cases use DeltaSource#forBoundedRowData utility method to instantiate the source. After instantiation of this builder you can either call RowDataBoundedDeltaSourceBuilder#build() method to get the instance of a DeltaSource or configure additional options using builder's API.

Linear Supertypes
BoundedDeltaSourceBuilder[RowData, RowDataBoundedDeltaSourceBuilder], DeltaSourceBuilderBase[RowData, RowDataBoundedDeltaSourceBuilder], AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. RowDataBoundedDeltaSourceBuilder
  2. BoundedDeltaSourceBuilder
  3. DeltaSourceBuilderBase
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def build[V <: DeltaSource[RowData]](): V
    Definition Classes
    DeltaSourceBuilderBase

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def build(): DeltaSource[RowData]

    Creates an instance of DeltaSource for a stream of RowData.

    Creates an instance of DeltaSource for a stream of RowData. Created source will work in Bounded mode, meaning it will read the content of the configured Delta snapshot at the fixed version, ignoring all changes done to this table after starting this source.

    This method can throw DeltaSourceValidationException in case of invalid arguments passed to Delta source builder.

    returns

    New DeltaSource instance.

    Annotations
    @Override() @SuppressWarnings()
  6. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  7. def columnNames(columnNames: <repeated...>[String]): RowDataBoundedDeltaSourceBuilder

    Specifies an array of column names that should be read from Delta table.

    Specifies an array of column names that should be read from Delta table. If this method is not used, Source will read all columns from Delta table.

    If provided List is null or contains null, empty or blank elements it will throw a DeltaSourceValidationException by builder after calling build() method.

    columnNames

    column names that should be read.

  8. def columnNames(columnNames: List[String]): RowDataBoundedDeltaSourceBuilder

    Specifies a List of column names that should be read from Delta table.

    Specifies a List of column names that should be read from Delta table. If this method is not used, Source will read all columns from Delta table.

    If provided List is null or contains null, empty or blank elements it will throw a DeltaSourceValidationException by builder after calling build() method.

    columnNames

    column names that should be read.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. def getApplicableOptions(): Collection[String]
    Attributes
    protected[builder]
    Definition Classes
    BoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  13. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. def getSourceConfiguration(): DeltaConnectorConfiguration

    returns

    A copy of DeltaConnectorConfiguration used by builder. The changes made on returned copy do not change the state of builder's configuration.

    Definition Classes
    DeltaSourceBuilderBase
  15. def getSourceSchema(): SourceSchema

    Extracts Delta table schema from DeltaLog io.delta.standalone.actions.Metadata including column names and column types converted to org.apache.flink.table.types.logical.LogicalType.

    Extracts Delta table schema from DeltaLog io.delta.standalone.actions.Metadata including column names and column types converted to org.apache.flink.table.types.logical.LogicalType.

    If #userColumnNames were defined, only those columns will be included in extracted schema.

    returns

    A SourceSchema including Delta table column names with their types that should be read from Delta table.

    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  16. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  19. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  20. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. def option(optionName: String, optionValue: Long): RowDataBoundedDeltaSourceBuilder

    Sets a configuration option.

    Sets a configuration option.

    optionName

    Option name to set.

    optionValue

    Option long value to set.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  22. def option(optionName: String, optionValue: Int): RowDataBoundedDeltaSourceBuilder

    Sets a configuration option.

    Sets a configuration option.

    optionName

    Option name to set.

    optionValue

    Option int value to set.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  23. def option(optionName: String, optionValue: Boolean): RowDataBoundedDeltaSourceBuilder

    Sets a configuration option.

    Sets a configuration option.

    optionName

    Option name to set.

    optionValue

    Option boolean value to set.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  24. def option(optionName: String, optionValue: String): RowDataBoundedDeltaSourceBuilder

    Sets a configuration option.

    Sets a configuration option.

    optionName

    Option name to set.

    optionValue

    Option String value to set.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  25. def prepareInapplicableOptionMessage(usedOptions: Collection[String], applicableOptions: Collection[String]): String
    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  26. def prepareOptionExclusionMessage(mutualExclusiveOptions: <repeated...>[String]): String
    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  27. def self(): RowDataBoundedDeltaSourceBuilder
    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
    Annotations
    @SuppressWarnings()
  28. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  29. def timestampAsOf(snapshotTimestamp: String): RowDataBoundedDeltaSourceBuilder

    Sets value of "timestampAsOf" option.

    Sets value of "timestampAsOf" option. With this option we will load the latest table version that was generated at or before the given timestamp.

    This option is mutually exclusive with #versionAsOf(long) option.

    snapshotTimestamp

    The timestamp we should time travel to. Supported formats are:

    • 2022-02-24
    • 2022-02-24 04:55:00
    • 2022-02-24 04:55:00.001
    • 2022-02-24T04:55:00
    • 2022-02-24T04:55:00.001
    • 2022-02-24T04:55:00.001Z
    Definition Classes
    RowDataBoundedDeltaSourceBuilder → BoundedDeltaSourceBuilder
    Annotations
    @Override()
  30. def toString(): String
    Definition Classes
    AnyRef → Any
  31. def validate(): Unit

    Validate definition of Delta source builder including mandatory and optional options.

    Validate definition of Delta source builder including mandatory and optional options.

    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  32. def validateInapplicableOptions(): Validator

    Validated builder options that were used but they might be not applicable for given builder type, for example using options from bounded mode like "versionAsOf" for continuous mode builder.

    Validated builder options that were used but they might be not applicable for given builder type, for example using options from bounded mode like "versionAsOf" for continuous mode builder.

    returns

    The Validator object with all (if any) validation error messages.

    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  33. def validateMandatoryOptions(): Validator
    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  34. def validateOptionExclusions(): Validator

    This method should implement any logic for validation of mutually exclusive options.

    This method should implement any logic for validation of mutually exclusive options.

    returns

    Validator instance with validation error message.

    Attributes
    protected[builder]
    Definition Classes
    BoundedDeltaSourceBuilder → DeltaSourceBuilderBase
    Annotations
    @Override()
  35. def validateOptionalParameters(): Validator
    Attributes
    protected[builder]
    Definition Classes
    DeltaSourceBuilderBase
  36. def versionAsOf(snapshotVersion: Long): RowDataBoundedDeltaSourceBuilder

    Sets value of "versionAsOf" option.

    Sets value of "versionAsOf" option. With this option we will load the given table version and read from it.

    This option is mutually exclusive with #timestampAsOf(String) option.

    snapshotVersion

    Delta table version to time travel to.

    Definition Classes
    RowDataBoundedDeltaSourceBuilder → BoundedDeltaSourceBuilder
    Annotations
    @Override()
  37. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  39. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from BoundedDeltaSourceBuilder[RowData, RowDataBoundedDeltaSourceBuilder]

Inherited from DeltaSourceBuilderBase[RowData, RowDataBoundedDeltaSourceBuilder]

Inherited from AnyRef

Inherited from Any

Ungrouped