Packages

trait OptimizeTableStrategy extends AnyRef

Defines set of utilities used in OptimizeTableCommand. The behavior of these utilities will change based on the OptimizeTableMode: COMPACTION, ZORDER and CLUSTERING.

Linear Supertypes
AnyRef, Any
Known Subclasses
ClusteringStrategy, CompactionStrategy, ZOrderStrategy
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OptimizeTableStrategy
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def curve: String

    The clustering algorithm to be used by either by ZORDER or Liquid CLUSTERING.

    The clustering algorithm to be used by either by ZORDER or Liquid CLUSTERING. An error is thrown for COMPACTION.

  2. abstract def maxBinSize: Long

    Utility method to get max bin size in bytes to group files into.

  3. abstract def optimizeTableMode: OptimizeTableMode.Value

    The optimize mode the strategy instance is created for.

  4. abstract def sparkSession: SparkSession
  5. abstract def updateOptimizeStats(optimizeStats: OptimizeStats, removedFiles: Seq[RemoveFile], bins: Seq[Bin]): Unit

    Utility to update additional metrics after optimization.

    Utility to update additional metrics after optimization.

    optimizeStats

    The input stats to update on.

    removedFiles

    Removed files.

    bins

    Sequence of bin-packed file groups, where each group consists of a partition value and its associated files.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  7. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  8. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  9. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  10. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def initNewBin: BinInfo

    Prepare a new Bin and returns its initialized BinInfo.

    Prepare a new Bin and returns its initialized BinInfo.

    This function is expected to be called once for each bin before tagAddFile is called.

  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. def prepareFilesPerPartition(inputFiles: Seq[AddFile]): Seq[AddFile]

    Utility method to prepare files in a partition for optimization.

    Utility method to prepare files in a partition for optimization.

    By default it sorts files on the size for the binpack.

    returns

    Prepared files for the subsequent optimization.

  17. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  18. def tagAddFile(file: AddFile, binInfo: BinInfo): AddFile

    Incorporate essential tags for optimized files based on the OptimizeTableMode.

  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from AnyRef

Inherited from Any

Ungrouped