class DeltaOptions extends DeltaWriteOptions with DeltaReadOptions with Serializable
Options for the Delta data source.
- Alphabetic
- By Inheritance
- DeltaOptions
- Serializable
- Serializable
- DeltaReadOptions
- DeltaWriteOptions
- DeltaWriteOptionsImpl
- DeltaOptionParser
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
canMergeSchema: Boolean
Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration.
Whether the user has enabled auto schema merging in writes using either a DataFrame option or SQL Session configuration. Automerging is off when table ACLs are enabled. We always respect the DataFrame writer configuration over the session config.
- Definition Classes
- DeltaWriteOptionsImpl
-
def
canOverwriteSchema: Boolean
Whether to allow overwriting the schema of a Delta table in an overwrite mode operation.
Whether to allow overwriting the schema of a Delta table in an overwrite mode operation. If ACLs are enabled, we can't change the schema of an operation through a write, which requires MODIFY permissions, when schema changes require OWN permissions.
- Definition Classes
- DeltaWriteOptionsImpl
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
containsStartingVersionOrTimestamp: Boolean
- Definition Classes
- DeltaReadOptions
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
val
excludeRegex: Option[Regex]
- Definition Classes
- DeltaReadOptions
-
val
failOnDataLoss: Boolean
- Definition Classes
- DeltaReadOptions
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
val
ignoreChanges: Boolean
- Definition Classes
- DeltaReadOptions
-
val
ignoreDeletes: Boolean
- Definition Classes
- DeltaReadOptions
-
val
ignoreFileDeletion: Boolean
- Definition Classes
- DeltaReadOptions
-
def
isDynamicPartitionOverwriteMode: Boolean
Whether to only overwrite partitions that have data written into it at runtime.
Whether to only overwrite partitions that have data written into it at runtime.
- Definition Classes
- DeltaWriteOptionsImpl
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
val
maxBytesPerTrigger: Option[Long]
- Definition Classes
- DeltaReadOptions
-
val
maxFilesPerTrigger: Option[Int]
- Definition Classes
- DeltaReadOptions
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
val
optimizeWrite: Option[Boolean]
Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.
Whether to add an adaptive shuffle before writing out the files to break skew, and coalesce data into chunkier files.
- Definition Classes
- DeltaWriteOptions
-
val
options: CaseInsensitiveMap[String]
- Attributes
- protected[delta]
- Definition Classes
- DeltaOptions → DeltaOptionParser
-
val
partitionOverwriteModeInOptions: Boolean
Whether partitionOverwriteMode is provided as a DataFrameWriter option.
Whether partitionOverwriteMode is provided as a DataFrameWriter option.
- Definition Classes
- DeltaWriteOptionsImpl
-
val
readChangeFeed: Boolean
- Definition Classes
- DeltaReadOptions
-
def
rearrangeOnly: Boolean
Whether to write new data to the table or just rearrange data that is already part of the table.
Whether to write new data to the table or just rearrange data that is already part of the table. This option declares that the data being written by this job does not change any data in the table and merely rearranges existing data. This makes sure streaming queries reading from this table will not see any new changes
- Definition Classes
- DeltaWriteOptionsImpl
-
val
replaceWhere: Option[String]
- Definition Classes
- DeltaWriteOptions
-
val
schemaTrackingLocation: Option[String]
- Definition Classes
- DeltaReadOptions
-
val
skipChangeCommits: Boolean
- Definition Classes
- DeltaReadOptions
-
val
sourceTrackingId: Option[String]
- Definition Classes
- DeltaReadOptions
-
val
sqlConf: SQLConf
- Attributes
- protected
- Definition Classes
- DeltaOptions → DeltaOptionParser
-
val
startingTimestamp: Option[String]
- Definition Classes
- DeltaReadOptions
-
val
startingVersion: Option[DeltaStartingVersion]
- Definition Classes
- DeltaReadOptions
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toBoolean(input: String, name: String): Boolean
- Definition Classes
- DeltaOptionParser
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
val
txnAppId: Option[String]
- Definition Classes
- DeltaWriteOptionsImpl
-
val
txnVersion: Option[Long]
- Definition Classes
- DeltaWriteOptionsImpl
-
val
userMetadata: Option[String]
- Definition Classes
- DeltaWriteOptions
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()