t

org.apache.spark.sql.delta.actions

TableFeatureSupport

trait TableFeatureSupport extends AnyRef

Trait to be mixed into the Protocol case class to enable Table Features.

Protocol reader version 3 and writer version 7 start to support reader and writer table features. Reader version 3 supports only reader-writer features in an explicit way, by adding its name to readerFeatures. Similarly, writer version 7 supports only writer-only or reader-writer features in an explicit way, by adding its name to writerFeatures. When reading or writing a table, clients MUST respect all supported features.

See also the document of TableFeature for feature-specific terminologies.

Self Type
Protocol
Linear Supertypes
AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TableFeatureSupport
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def canDowngradeTo(to: Protocol, droppedFeatureName: String): Boolean

    Determine whether this protocol can be safely downgraded to a new protocol to.

    Determine whether this protocol can be safely downgraded to a new protocol to. All the implicit and explicit features between the two protocols need to match, excluding the dropped feature. We also need to take into account that in some cases the downgrade process may add the CheckpointProtectionTableFeature.

    Note, the conditions above also account for cases where we downgrade from table features to legacy protocol versions.

  6. def canTransitionTo(to: Protocol, op: Operation): Boolean

    True if this protocol can be upgraded or downgraded to the 'to' protocol.

  7. def canUpgradeTo(to: Protocol): Boolean

    Determine whether this protocol can be safely upgraded to a new protocol to.

    Determine whether this protocol can be safely upgraded to a new protocol to. This means:

    • all features supported by this protocol are supported by to.

    Examples regarding feature status:

    • from [appendOnly] to [appendOnly] => allowed.
    • from [appendOnly, changeDataFeed] to [appendOnly] => not allowed.
    • from [appendOnly] to [appendOnly, changeDataFeed] => allowed.
  8. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  9. def denormalized: Protocol

    Protocol denormalization is the process of converting a legacy protocol to the the equivalent table features protocol.

    Protocol denormalization is the process of converting a legacy protocol to the the equivalent table features protocol. This is the inverse of protocol normalization. It can be used to allow operations on legacy protocols that yield result which cannot be represented anymore by a legacy protocol.

  10. def denormalizedNormalized: Protocol

    Helper method that applies both denormalization and normalization.

    Helper method that applies both denormalization and normalization. This can be used to normalize invalid legacy protocols such as (2, 3), (1, 5). A legacy protocol is invalid when the version numbers are higher than required to support the implied feature set.

  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. lazy val implicitlyAndExplicitlySupportedFeatures: Set[TableFeature]

    Get all features that are supported by this protocol, implicitly and explicitly.

    Get all features that are supported by this protocol, implicitly and explicitly. When the protocol supports table features, this method returns the same set of features as readerAndWriterFeatureNames; when the protocol does not support table features, this method becomes equivalent to implicitlySupportedFeatures.

    Annotations
    @JsonIgnore()
  17. lazy val implicitlySupportedFeatures: Set[TableFeature]

    Get all features that are implicitly supported by this protocol, for example, Protocol(1,2) implicitly supports appendOnly and invariants.

    Get all features that are implicitly supported by this protocol, for example, Protocol(1,2) implicitly supports appendOnly and invariants. When this protocol is capable of requiring writer features, no feature can be implicitly supported.

    Annotations
    @JsonIgnore()
  18. def isFeatureSupported(feature: TableFeature): Boolean

    Check if a feature is supported by this protocol.

    Check if a feature is supported by this protocol. This means either (a) the protocol does not support table features and implicitly supports the feature, or (b) the protocol supports table features and references the feature.

  19. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  20. def merge(others: Protocol*): Protocol

    Merge this protocol with multiple protocols to have the highest reader and writer versions plus all explicitly and implicitly supported features.

  21. lazy val nativeReaderAndWriterFeatures: Seq[TableFeature]

    A sequence of native TableFeatures.

    A sequence of native TableFeatures. This is derived by filtering out all explicitly supported legacy features.

    Annotations
    @JsonIgnore()
  22. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  23. def normalized: Protocol

    Protocol normalization is the process of converting a table features protocol to the weakest possible form.

    Protocol normalization is the process of converting a table features protocol to the weakest possible form. This primarily refers to converting a table features protocol to a legacy protocol. A Table Features protocol can be represented with the legacy representation only when the features set of the former exactly matches a legacy protocol.

    Normalization can also decrease the reader version of a table features protocol when it is higher than necessary.

    For example: (1, 7, AppendOnly, Invariants, CheckConstraints) -> (1, 3) (3, 7, RowTracking) -> (1, 7, RowTracking)

  24. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  25. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  26. lazy val readerAndWriterFeatureNames: Set[String]

    Get a set of all feature names in this protocol's readerFeatures and writerFeatures field.

    Get a set of all feature names in this protocol's readerFeatures and writerFeatures field. Returns an empty set when this protocol supports none of reader and writer features.

    Annotations
    @JsonIgnore()
  27. lazy val readerAndWriterFeatures: Seq[TableFeature]

    Same as above but returns a sequence of TableFeature instead of a set of feature names.

    Same as above but returns a sequence of TableFeature instead of a set of feature names.

    Annotations
    @JsonIgnore()
  28. def readerFeatureNames: Set[String]

    Get all feature names in this protocol's readerFeatures field.

    Get all feature names in this protocol's readerFeatures field. Returns an empty set when this protocol does not support reader features.

  29. def removeFeature(targetFeature: TableFeature): Protocol

    Remove feature wrapper for removing either Reader/Writer or Writer features.

    Remove feature wrapper for removing either Reader/Writer or Writer features. We assume the feature exists in the protocol. There is a relevant validation at AlterTableDropFeatureDeltaCommand. We also require targetFeature is removable.

    After removing the feature we normalize the protocol.

  30. def supportsReaderFeatures: Boolean

    Check if this protocol is capable of adding features into its readerFeatures field.

  31. def supportsWriterFeatures: Boolean

    Check if this protocol is capable of adding features into its writerFeatures field.

  32. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  33. def toString(): String
    Definition Classes
    AnyRef → Any
  34. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  35. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  36. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  37. def withFeature(feature: TableFeature): Protocol

    Get a new Protocol object that has feature supported.

    Get a new Protocol object that has feature supported. Writer-only features will be added to writerFeatures field, and reader-writer features will be added to readerFeatures and writerFeatures fields.

    If feature is already implicitly supported in the current protocol's legacy reader or writer protocol version, the new protocol will not modify the original protocol version, i.e., the feature will not be explicitly added to the protocol's readerFeatures or writerFeatures. This is to avoid unnecessary protocol upgrade for feature that it already supports.

  38. def withFeatures(features: Iterable[TableFeature]): Protocol

    Get a new Protocol object with multiple features supported.

    Get a new Protocol object with multiple features supported.

    See the documentation of withFeature for more information.

  39. def writerFeatureNames: Set[String]

    Get a set of all feature names in this protocol's writerFeatures field.

    Get a set of all feature names in this protocol's writerFeatures field. Returns an empty set when this protocol does not support writer features.

Inherited from AnyRef

Inherited from Any

Ungrouped