case class CompactionStrategy(sparkSession: SparkSession, optimizeContext: DeltaOptimizeContext) extends OptimizeTableStrategy with Product with Serializable
Implements compaction strategy
- Alphabetic
- By Inheritance
- CompactionStrategy
- Serializable
- Serializable
- Product
- Equals
- OptimizeTableStrategy
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
- new CompactionStrategy(sparkSession: SparkSession, optimizeContext: DeltaOptimizeContext)
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
curve: String
The clustering algorithm to be used by either by ZORDER or Liquid CLUSTERING.
The clustering algorithm to be used by either by ZORDER or Liquid CLUSTERING. An error is thrown for COMPACTION.
- Definition Classes
- CompactionStrategy → OptimizeTableStrategy
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
initNewBin: BinInfo
Prepare a new Bin and returns its initialized BinInfo.
Prepare a new Bin and returns its initialized BinInfo.
This function is expected to be called once for each bin before tagAddFile is called.
- Definition Classes
- OptimizeTableStrategy
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
val
maxBinSize: Long
Utility method to get max bin size in bytes to group files into.
Utility method to get max bin size in bytes to group files into.
- Definition Classes
- CompactionStrategy → OptimizeTableStrategy
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val optimizeContext: DeltaOptimizeContext
-
val
optimizeTableMode: OptimizeTableMode.Value
The optimize mode the strategy instance is created for.
The optimize mode the strategy instance is created for.
- Definition Classes
- CompactionStrategy → OptimizeTableStrategy
-
def
prepareFilesPerPartition(inputFiles: Seq[AddFile]): Seq[AddFile]
Utility method to prepare files in a partition for optimization.
Utility method to prepare files in a partition for optimization.
By default it sorts files on the size for the binpack.
- returns
Prepared files for the subsequent optimization.
- Definition Classes
- OptimizeTableStrategy
-
val
sparkSession: SparkSession
- Definition Classes
- CompactionStrategy → OptimizeTableStrategy
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
tagAddFile(file: AddFile, binInfo: BinInfo): AddFile
Incorporate essential tags for optimized files based on the OptimizeTableMode.
Incorporate essential tags for optimized files based on the OptimizeTableMode.
- Definition Classes
- OptimizeTableStrategy
-
def
updateOptimizeStats(optimizeStats: OptimizeStats, removedFiles: Seq[RemoveFile], bins: Seq[Bin]): Unit
Utility to update additional metrics after optimization.
Utility to update additional metrics after optimization.
- optimizeStats
The input stats to update on.
- removedFiles
Removed files.
- bins
Sequence of bin-packed file groups, where each group consists of a partition value and its associated files.
- Definition Classes
- CompactionStrategy → OptimizeTableStrategy
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()