t

org.apache.spark.sql.delta.actions

TableFeatureSupport

trait TableFeatureSupport extends AnyRef

Trait to be mixed into the Protocol case class to enable Table Features.

Protocol reader version 3 and writer version 7 start to support reader and writer table features. Reader version 3 supports only reader-writer features in an explicit way, by adding its name to readerFeatures. Similarly, writer version 7 supports only writer-only or reader-writer features in an explicit way, by adding its name to writerFeatures. When reading or writing a table, clients MUST respect all supported features.

See also the document of TableFeature for feature-specific terminologies.

Self Type
Protocol
Linear Supertypes
AnyRef, Any
Known Subclasses
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. TableFeatureSupport
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def canDowngradeTo(to: Protocol, droppedFeatureName: String): Boolean

    Determine whether this protocol can be safely downgraded to a new protocol to.

    Determine whether this protocol can be safely downgraded to a new protocol to. This includes the following:

    • The current protocol needs to support at least writer features. This is because protocol downgrade is only supported with table features.
    • The protocol version can only be downgraded when there are no non-legacy table features.
    • We can only remove one feature at a time.
    • When downgrading protocol versions, the resulting versions must support exactly the same set of legacy features supported by the current protocol.

    Note, this not an exhaustive list of downgrade rules. Rather, we check the most important downgrade invariants. We also perform checks during feature removal at AlterTableDropFeatureDeltaCommand.

  6. def canTransitionTo(to: Protocol, op: Operation): Boolean

    True if this protocol can be upgraded or downgraded to the 'to' protocol.

  7. def canUpgradeTo(to: Protocol): Boolean

    Determine whether this protocol can be safely upgraded to a new protocol to.

    Determine whether this protocol can be safely upgraded to a new protocol to. This means:

    • this protocol has reader protocol version less than or equals to to.
    • this protocol has writer protocol version less than or equals to to.
    • all features supported by this protocol are supported by to.

    Examples regarding feature status:

    • from [appendOnly] to [appendOnly] => allowed
    • from [appendOnly, changeDataFeed] to [appendOnly] => not allowed
    • from [appendOnly] to [appendOnly, changeDataFeed] => allowed
  8. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  9. def downgradeProtocolVersionsIfNeeded: Protocol

    If the current protocol does not contain any non-legacy table features and the remaining set of legacy table features exactly matches a legacy protocol version, it downgrades the protocol to the minimum reader/writer versions required to support the protocol's legacy features.

    If the current protocol does not contain any non-legacy table features and the remaining set of legacy table features exactly matches a legacy protocol version, it downgrades the protocol to the minimum reader/writer versions required to support the protocol's legacy features.

    Note, when a table is initialized with table features (3, 7), by default there are no legacy features. After we remove the last native feature we downgrade the protocol to (1, 1).

  10. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  11. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  12. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  13. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. lazy val implicitlyAndExplicitlySupportedFeatures: Set[TableFeature]

    Get all features that are supported by this protocol, implicitly and explicitly.

    Get all features that are supported by this protocol, implicitly and explicitly. When the protocol supports table features, this method returns the same set of features as readerAndWriterFeatureNames; when the protocol does not support table features, this method becomes equivalent to implicitlySupportedFeatures.

    Annotations
    @JsonIgnore()
  16. lazy val implicitlySupportedFeatures: Set[TableFeature]

    Get all features that are implicitly supported by this protocol, for example, Protocol(1,2) implicitly supports appendOnly and invariants.

    Get all features that are implicitly supported by this protocol, for example, Protocol(1,2) implicitly supports appendOnly and invariants. When this protocol is capable of requiring writer features, no feature can be implicitly supported.

    Annotations
    @JsonIgnore()
  17. def isFeatureSupported(feature: TableFeature): Boolean

    Check if a feature is supported by this protocol.

    Check if a feature is supported by this protocol. This means either (a) the protocol does not support table features and implicitly supports the feature, or (b) the protocol supports table features and references the feature.

  18. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  19. def merge(others: Protocol*): Protocol

    Merge this protocol with multiple protocols to have the highest reader and writer versions plus all explicitly and implicitly supported features.

  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. lazy val readerAndWriterFeatureNames: Set[String]

    Get a set of all feature names in this protocol's readerFeatures and writerFeatures field.

    Get a set of all feature names in this protocol's readerFeatures and writerFeatures field. Returns an empty set when this protocol supports none of reader and writer features.

    Annotations
    @JsonIgnore()
  24. lazy val readerAndWriterFeatures: Seq[TableFeature]

    Same as above but returns a sequence of TableFeature instead of a set of feature names.

    Same as above but returns a sequence of TableFeature instead of a set of feature names.

    Annotations
    @JsonIgnore()
  25. def readerFeatureNames: Set[String]

    Get all feature names in this protocol's readerFeatures field.

    Get all feature names in this protocol's readerFeatures field. Returns an empty set when this protocol does not support reader features.

  26. def removeFeature(targetFeature: TableFeature): Protocol

    Remove feature wrapper for removing either Reader/Writer or Writer features.

    Remove feature wrapper for removing either Reader/Writer or Writer features. We assume the feature exists in the protocol. There is a relevant validation at AlterTableDropFeatureDeltaCommand. We also require targetFeature is removable.

    When the feature to remove is the last explicit table feature of the table we also remove the TableFeatures feature and downgrade the protocol.

  27. def supportsReaderFeatures: Boolean

    Check if this protocol is capable of adding features into its readerFeatures field.

  28. def supportsWriterFeatures: Boolean

    Check if this protocol is capable of adding features into its writerFeatures field.

  29. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  30. def toString(): String
    Definition Classes
    AnyRef → Any
  31. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  32. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  33. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  34. def withFeature(feature: TableFeature): Protocol

    Get a new Protocol object that has feature supported.

    Get a new Protocol object that has feature supported. Writer-only features will be added to writerFeatures field, and reader-writer features will be added to readerFeatures and writerFeatures fields.

    If feature is already implicitly supported in the current protocol's legacy reader or writer protocol version, the new protocol will not modify the original protocol version, i.e., the feature will not be explicitly added to the protocol's readerFeatures or writerFeatures. This is to avoid unnecessary protocol upgrade for feature that it already supports.

  35. def withFeatures(features: Iterable[TableFeature]): Protocol

    Get a new Protocol object with multiple features supported.

    Get a new Protocol object with multiple features supported.

    See the documentation of withFeature for more information.

  36. def writerFeatureNames: Set[String]

    Get a set of all feature names in this protocol's writerFeatures field.

    Get a set of all feature names in this protocol's writerFeatures field. Returns an empty set when this protocol does not support writer features.

Inherited from AnyRef

Inherited from Any

Ungrouped