object InCommitTimestampTableFeature extends WriterFeature with FeatureAutomaticallyEnabledByMetadata with RemovableFeature
inCommitTimestamp table feature is a writer feature that makes every writer write a monotonically increasing timestamp inside the commit file.
- Alphabetic
- By Inheritance
- InCommitTimestampTableFeature
- RemovableFeature
- FeatureAutomaticallyEnabledByMetadata
- WriterFeature
- TableFeature
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def actionUsesFeature(action: Action): Boolean
- Definition Classes
- InCommitTimestampTableFeature → RemovableFeature
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def automaticallyUpdateProtocolOfExistingTables: Boolean
Whether the feature can automatically update the protocol of an existing table when the metadata requirements are satisfied.
Whether the feature can automatically update the protocol of an existing table when the metadata requirements are satisfied. As a rule of thumb, a table feature that requires explicit operations (e.g., turning on a table property) should set this flag to
true, while features that are used implicitly (e.g., when using a new data type) should set this flag tofalse.- Definition Classes
- InCommitTimestampTableFeature → FeatureAutomaticallyEnabledByMetadata
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def historyContainsFeature(spark: SparkSession, downgradeTxnReadSnapshot: Snapshot): Boolean
Examines all historical commits for traces of the removableFeature.
Examines all historical commits for traces of the removableFeature. This is achieved as follows:
1) We find the earliest valid checkpoint, recreate a snapshot at that version and we check whether there any traces of the feature-to-remove. 2) We check all commits that exist between version 0 and the current version. This includes the versions we validated the snapshots. This is because a commit might include information that is not available in the snapshot. Examples include CommitInfo, CDCInfo etc. Note, there can still be valid log commit files with versions prior the earliest checkpoint version. 3) We do not need to recreate a snapshot at the current version because this is already being handled by validateRemoval.
Note, this is a slow process.
- spark
The SparkSession.
- downgradeTxnReadSnapshot
The read snapshot of the protocol downgrade transaction.
- returns
True if the history contains any trace of the feature.
- Definition Classes
- RemovableFeature
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isLegacyFeature: Boolean
Determine if this feature is a legacy feature.
Determine if this feature is a legacy feature. See the documentation of TableFeature for more information.
- Definition Classes
- TableFeature
- def isReaderWriterFeature: Boolean
Determine if this feature applies to both readers and writers.
Determine if this feature applies to both readers and writers.
- Definition Classes
- TableFeature
- def isRemovable: Boolean
True if this feature can be removed.
True if this feature can be removed.
- Definition Classes
- TableFeature
- def metadataRequiresFeatureToBeEnabled(protocol: Protocol, metadata: Metadata, spark: SparkSession): Boolean
Determine whether the feature must be supported and enabled because its metadata requirements are satisfied.
Determine whether the feature must be supported and enabled because its metadata requirements are satisfied.
- Definition Classes
- InCommitTimestampTableFeature → FeatureAutomaticallyEnabledByMetadata
- def minProtocolVersion: Protocol
Get a Protocol object stating the minimum reader and writer versions this feature requires.
Get a Protocol object stating the minimum reader and writer versions this feature requires. For a feature that can only be explicitly supported, this method returns a protocol version that supports table features, either
(0,7)or(3,7)depending on the feature is writer-only or reader-writer. For a legacy feature that can be implicitly supported, this method returns the first protocol version which introduced the said feature.For all features, if the table's protocol version does not support table features, then the minimum protocol version is enough. However, if the protocol version supports table features for the feature type (writer-only or reader-writer), then the minimum protocol version is not enough to support a feature. In this case the feature must also be explicitly listed in the appropriate feature sets in the Protocol.
- Definition Classes
- TableFeature
- val minReaderVersion: Int
- Definition Classes
- TableFeature
- val minWriterVersion: Int
- Definition Classes
- TableFeature
- val name: String
- Definition Classes
- TableFeature
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- def preDowngradeCommand(table: DeltaTableV2): PreDowngradeTableFeatureCommand
- Definition Classes
- InCommitTimestampTableFeature → RemovableFeature
- def requiredFeatures: Set[TableFeature]
Set of table features that this table feature depends on.
Set of table features that this table feature depends on. I.e. the set of features that need to be enabled if this table feature is enabled.
- Definition Classes
- TableFeature
- def requiresHistoryProtection: Boolean
- Definition Classes
- RemovableFeature
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def validateRemoval(snapshot: Snapshot): Boolean
As per the spec, we can disable ICT by just setting DeltaConfigs.IN_COMMIT_TIMESTAMPS_ENABLED to
false.As per the spec, we can disable ICT by just setting DeltaConfigs.IN_COMMIT_TIMESTAMPS_ENABLED to
false. There is no need to remove the provenance properties. However, InCommitTimestampsPreDowngradeCommand will try to remove these properties because they can be removed as part of the same metadata update that sets DeltaConfigs.IN_COMMIT_TIMESTAMPS_ENABLED tofalse. We check all three properties here as well for consistency.- Definition Classes
- InCommitTimestampTableFeature → RemovableFeature
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()