class DeltaIdentityColumnStatsTracker extends DeltaJobStatisticsTracker
Stats tracker for IDENTITY column high water marks. The only difference between this class and
DeltaJobStatisticsTracker is how the stats are aggregated on the driver.
Linear Supertypes
Ordering
- Alphabetic
- By Inheritance
Inherited
- DeltaIdentityColumnStatsTracker
- DeltaJobStatisticsTracker
- WriteJobStatsTracker
- Serializable
- AnyRef
- Any
- Hide All
- Show All
Visibility
- Public
- Protected
Instance Constructors
- new DeltaIdentityColumnStatsTracker(hadoopConf: Configuration, path: Path, dataCols: Seq[Attribute], statsColExpr: Expression, identityColumnInfo: Seq[(String, Boolean)])
- hadoopConf
The Hadoop configuration object to use on an executor.
- path
Root Reservoir path
- dataCols
Resolved data (i.e. non-partitionBy) columns of the dataframe to be written.
- statsColExpr
The expression to collect high water marks.
- identityColumnInfo
Information of IDENTITY columns. It contains a pair of column name and whether it has a positive step for each IDENTITY column.
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- val dataCols: Seq[Attribute]
- Definition Classes
- DeltaJobStatisticsTracker
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- var highWaterMarks: Map[String, Long]
- val identityColumnInfo: Seq[(String, Boolean)]
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def newTaskInstance(): WriteTaskStatsTracker
- Definition Classes
- DeltaJobStatisticsTracker → WriteJobStatsTracker
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- val path: Path
- Definition Classes
- DeltaJobStatisticsTracker
- def processStats(stats: Seq[WriteTaskStats], jobCommitTime: Long): Unit
- Definition Classes
- DeltaIdentityColumnStatsTracker → DeltaJobStatisticsTracker → WriteJobStatsTracker
- var recordedStats: Map[String, String]
- Definition Classes
- DeltaJobStatisticsTracker
- val statsColExpr: Expression
- Definition Classes
- DeltaJobStatisticsTracker
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()