object VacuumProtocolCheckTableFeature extends ReaderWriterFeature with RemovableFeature
A ReaderWriter table feature for VACUUM. If this feature is enabled: A writer should follow one of the following:
- Non-Support for Vacuum: Writers can explicitly state that they do not support VACUUM for any table, regardless of whether the Vacuum Protocol Check Table feature exists. 2. Implement Writer Protocol Check: Ensure that the VACUUM implementation includes a writer protocol check before any file deletions occur. Readers don't need to understand or change anything new; they just need to acknowledge the feature exists
- Alphabetic
- By Inheritance
- VacuumProtocolCheckTableFeature
- RemovableFeature
- ReaderWriterFeature
- ReaderWriterFeatureType
- WriterFeature
- TableFeature
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
actionUsesFeature(action: Action): Boolean
- Definition Classes
- VacuumProtocolCheckTableFeature → RemovableFeature
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
historyContainsFeature(spark: SparkSession, downgradeTxnReadSnapshot: Snapshot): Boolean
Examines all historical commits for traces of the removableFeature.
Examines all historical commits for traces of the removableFeature. This is achieved as follows:
1) We find the earliest valid checkpoint, recreate a snapshot at that version and we check whether there any traces of the feature-to-remove. 2) We check all commits that exist between version 0 and the current version. This includes the versions we validated the snapshots. This is because a commit might include information that is not available in the snapshot. Examples include CommitInfo, CDCInfo etc. Note, there can still be valid log commit files with versions prior the earliest checkpoint version. 3) We do not need to recreate a snapshot at the current version because this is already being handled by validateRemoval.
Note, this is a slow process.
- spark
The SparkSession.
- downgradeTxnReadSnapshot
The read snapshot of the protocol downgrade transaction.
- returns
True if the history contains any trace of the feature.
- Definition Classes
- RemovableFeature
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
def
isLegacyFeature: Boolean
Determine if this feature is a legacy feature.
Determine if this feature is a legacy feature. See the documentation of TableFeature for more information.
- Definition Classes
- TableFeature
-
def
isReaderWriterFeature: Boolean
Determine if this feature applies to both readers and writers.
Determine if this feature applies to both readers and writers.
- Definition Classes
- TableFeature
-
def
isRemovable: Boolean
True if this feature can be removed.
True if this feature can be removed.
- Definition Classes
- TableFeature
-
def
minProtocolVersion: Protocol
Get a Protocol object stating the minimum reader and writer versions this feature requires.
Get a Protocol object stating the minimum reader and writer versions this feature requires. For a feature that can only be explicitly supported, this method returns a protocol version that supports table features, either
(0,7)or(3,7)depending on the feature is writer-only or reader-writer. For a legacy feature that can be implicitly supported, this method returns the first protocol version which introduced the said feature.For all features, if the table's protocol version does not support table features, then the minimum protocol version is enough. However, if the protocol version supports table features for the feature type (writer-only or reader-writer), then the minimum protocol version is not enough to support a feature. In this case the feature must also be explicitly listed in the appropriate feature sets in the Protocol.
- Definition Classes
- TableFeature
-
val
minReaderVersion: Int
- Definition Classes
- ReaderWriterFeature → TableFeature
-
val
minWriterVersion: Int
- Definition Classes
- TableFeature
-
val
name: String
- Definition Classes
- TableFeature
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
preDowngradeCommand(table: DeltaTableV2): PreDowngradeTableFeatureCommand
- Definition Classes
- VacuumProtocolCheckTableFeature → RemovableFeature
-
def
requiredFeatures: Set[TableFeature]
Set of table features that this table feature depends on.
Set of table features that this table feature depends on. I.e. the set of features that need to be enabled if this table feature is enabled.
- Definition Classes
- TableFeature
-
def
requiresHistoryProtection: Boolean
- Definition Classes
- RemovableFeature
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
validateRemoval(snapshot: Snapshot): Boolean
- Definition Classes
- VacuumProtocolCheckTableFeature → RemovableFeature
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()