trait RequiresDistributionAndOrdering extends Write
A write that requires a specific distribution and ordering of data.
- Annotations
- @Experimental()
- Since
3.2.0
- Alphabetic
- By Inheritance
- RequiresDistributionAndOrdering
- Write
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def requiredDistribution(): Distribution
Returns the distribution required by this write.
Returns the distribution required by this write.
Spark will distribute incoming records across partitions to satisfy the required distribution before passing the records to the data source table on write.
Implementations may return
UnspecifiedDistributionif they don't require any specific distribution of data on write.- returns
the required distribution
- abstract def requiredOrdering(): Array[SortOrder]
Returns the ordering required by this write.
Returns the ordering required by this write.
Spark will order incoming records within partitions to satisfy the required ordering before passing those records to the data source table on write.
Implementations may return an empty array if they don't require any specific ordering of data on write.
- returns
the required ordering
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- def description(): String
Returns the description associated with this write.
Returns the description associated with this write.
- Definition Classes
- Write
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def requiredNumPartitions(): Int
Returns the number of partitions required by this write.
Returns the number of partitions required by this write.
Implementations may override this to require a specific number of input partitions.
Note that Spark doesn't support the number of partitions on
UnspecifiedDistribution, the query will fail if the number of partitions are provided but the distribution is unspecified.- returns
the required number of partitions, any value less than 1 mean no requirement.
- def supportedCustomMetrics(): Array[CustomMetric]
Returns an array of supported custom metrics with name and description.
Returns an array of supported custom metrics with name and description. By default it returns empty array.
- Definition Classes
- Write
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toBatch(): BatchWrite
Returns a
BatchWriteto write data to batch source.Returns a
BatchWriteto write data to batch source. By default this method throws exception, data sources must overwrite this method to provide an implementation, if theTablethat creates this write returnsTableCapability#BATCH_WRITEsupport in itsTable#capabilities().- Definition Classes
- Write
- def toStreaming(): StreamingWrite
Returns a
StreamingWriteto write data to streaming source.Returns a
StreamingWriteto write data to streaming source. By default this method throws exception, data sources must overwrite this method to provide an implementation, if theTablethat creates this write returnsTableCapability#STREAMING_WRITEsupport in itsTable#capabilities().- Definition Classes
- Write
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()