Packages

class DeltaOptions extends DeltaWriteOptions with DeltaReadOptions with Serializable

Options for the Delta data source.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaOptions
  2. Serializable
  3. Serializable
  4. DeltaReadOptions
  5. DeltaWriteOptions
  6. DeltaWriteOptionsImpl
  7. DeltaOptionParser
  8. AnyRef
  9. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DeltaOptions(options: Map[String, String], conf: SQLConf)
  2. new DeltaOptions(options: CaseInsensitiveMap[String], sqlConf: SQLConf)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def canMergeSchema: Boolean

    Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration.

    Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration. Automerging is off when table ACLs are enabled. We always respect the DataFrame writer configuration over the session config.

    Definition Classes
    DeltaWriteOptionsImpl
  6. def canOverwriteSchema: Boolean

    Whether to allow overwriting the schema of a Delta table in an overwrite mode operation.

    Whether to allow overwriting the schema of a Delta table in an overwrite mode operation. If ACLs are enabled, we can't change the schema of an operation through a write, which requires MODIFY permissions, when schema changes require OWN permissions.

    Definition Classes
    DeltaWriteOptionsImpl
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  8. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  9. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  10. val excludeRegex: Option[Regex]
    Definition Classes
    DeltaReadOptions
  11. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  12. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. val ignoreChanges: Boolean
    Definition Classes
    DeltaReadOptions
  15. val ignoreDeletes: Boolean
    Definition Classes
    DeltaReadOptions
  16. val ignoreFileDeletion: Boolean
    Definition Classes
    DeltaReadOptions
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. val maxBytesPerTrigger: Option[Long]
    Definition Classes
    DeltaReadOptions
  19. val maxFilesPerTrigger: Option[Int]
    Definition Classes
    DeltaReadOptions
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. val optimizeWrite: Option[Boolean]

    Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.

    Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.

    Definition Classes
    DeltaWriteOptions
  24. val options: CaseInsensitiveMap[String]
    Attributes
    protected[delta]
    Definition Classes
    DeltaOptionsDeltaOptionParser
  25. def rearrangeOnly: Boolean

    Whether to write new data to the table or just rearrange data that is already part of the table.

    Whether to write new data to the table or just rearrange data that is already part of the table. This option declares that the data being written by this job does not change any data in the table and merely rearranges existing data. This makes sure streaming queries reading from this table will not see any new changes

    Definition Classes
    DeltaWriteOptionsImpl
  26. val replaceWhere: Option[String]
    Definition Classes
    DeltaWriteOptions
  27. val sqlConf: SQLConf
    Attributes
    protected
    Definition Classes
    DeltaOptionsDeltaOptionParser
  28. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  29. def toBoolean(input: String, name: String): Boolean
    Definition Classes
    DeltaOptionParser
  30. def toString(): String
    Definition Classes
    AnyRef → Any
  31. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  32. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  33. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from DeltaReadOptions

Inherited from DeltaWriteOptions

Inherited from DeltaWriteOptionsImpl

Inherited from DeltaOptionParser

Inherited from AnyRef

Inherited from Any

Ungrouped