case class UpdateCommand(tahoeFileIndex: TahoeFileIndex, target: LogicalPlan, updateExpressions: Seq[Expression], condition: Option[Expression]) extends LogicalPlan with RunnableCommand with DeltaCommand with Product with Serializable

Performs an Update using updateExpression on the rows that match condition

Algorithm: 1) Identify the affected files, i.e., the files that may have the rows to be updated. 2) Scan affected files, apply the updates, and generate a new DF with updated rows. 3) Use the Delta protocol to atomically write the new DF as new files and remove the affected files that are identified in step 1.

Linear Supertypes
Serializable, Serializable, DeltaCommand, DeltaLogging, DatabricksLogging, DeltaProgressReporter, RunnableCommand, Command, LogicalPlan, Logging, QueryPlanConstraints, ConstraintHelper, LogicalPlanStats, AnalysisHelper, QueryPlan[LogicalPlan], TreeNode[LogicalPlan], Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. UpdateCommand
  2. Serializable
  3. Serializable
  4. DeltaCommand
  5. DeltaLogging
  6. DatabricksLogging
  7. DeltaProgressReporter
  8. RunnableCommand
  9. Command
  10. LogicalPlan
  11. Logging
  12. QueryPlanConstraints
  13. ConstraintHelper
  14. LogicalPlanStats
  15. AnalysisHelper
  16. QueryPlan
  17. TreeNode
  18. Product
  19. Equals
  20. AnyRef
  21. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new UpdateCommand(tahoeFileIndex: TahoeFileIndex, target: LogicalPlan, updateExpressions: Seq[Expression], condition: Option[Expression])

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. lazy val allAttributes: AttributeSeq
    Definition Classes
    QueryPlan
  5. def analyzed: Boolean
    Definition Classes
    AnalysisHelper
  6. def apply(number: Int): TreeNode[_]
    Definition Classes
    TreeNode
  7. def argString: String
    Definition Classes
    TreeNode
  8. def asCode: String
    Definition Classes
    TreeNode
  9. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  10. def assertNotAnalysisRule(): Unit
    Attributes
    protected
    Definition Classes
    AnalysisHelper
  11. def buildBaseRelation(spark: SparkSession, txn: OptimisticTransaction, actionType: String, rootPath: Path, inputLeafFiles: Seq[String], nameToAddFileMap: Map[String, AddFile]): HadoopFsRelation

    Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.

    Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.

    Attributes
    protected
    Definition Classes
    DeltaCommand
  12. final lazy val canonicalized: LogicalPlan
    Definition Classes
    QueryPlan
    Annotations
    @transient()
  13. def children: Seq[LogicalPlan]
    Definition Classes
    Command → TreeNode
  14. def childrenResolved: Boolean
    Definition Classes
    LogicalPlan
  15. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  16. def collect[B](pf: PartialFunction[LogicalPlan, B]): Seq[B]
    Definition Classes
    TreeNode
  17. def collectFirst[B](pf: PartialFunction[LogicalPlan, B]): Option[B]
    Definition Classes
    TreeNode
  18. def collectLeaves(): Seq[LogicalPlan]
    Definition Classes
    TreeNode
  19. val condition: Option[Expression]
  20. def conf: SQLConf
    Definition Classes
    QueryPlan
  21. lazy val constraints: ExpressionSet
    Definition Classes
    QueryPlanConstraints
  22. def constructIsNotNullConstraints(constraints: Set[Expression], output: Seq[Attribute]): Set[Expression]
    Definition Classes
    ConstraintHelper
  23. lazy val containsChild: Set[TreeNode[_]]
    Definition Classes
    TreeNode
  24. def doCanonicalize(): LogicalPlan
    Attributes
    protected
    Definition Classes
    QueryPlan
  25. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  26. final def expressions: Seq[Expression]
    Definition Classes
    QueryPlan
  27. def fastEquals(other: TreeNode[_]): Boolean
    Definition Classes
    TreeNode
  28. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  29. def find(f: (LogicalPlan) ⇒ Boolean): Option[LogicalPlan]
    Definition Classes
    TreeNode
  30. def flatMap[A](f: (LogicalPlan) ⇒ TraversableOnce[A]): Seq[A]
    Definition Classes
    TreeNode
  31. def foreach(f: (LogicalPlan) ⇒ Unit): Unit
    Definition Classes
    TreeNode
  32. def foreachUp(f: (LogicalPlan) ⇒ Unit): Unit
    Definition Classes
    TreeNode
  33. def generateCandidateFileMap(basePath: Path, candidateFiles: Seq[AddFile]): Map[String, AddFile]

    Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update.

    Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update. We expect file names to be unique, because each file contains a UUID.

    Attributes
    protected
    Definition Classes
    DeltaCommand
  34. def generateTreeString(depth: Int, lastChildren: Seq[Boolean], builder: StringBuilder, verbose: Boolean, prefix: String, addSuffix: Boolean): StringBuilder
    Definition Classes
    TreeNode
  35. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  36. def getTouchedFile(basePath: Path, filePath: String, nameToAddFileMap: Map[String, AddFile]): AddFile

    Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.

    Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.

    filePath

    The path to a file. Can be either absolute or relative

    nameToAddFileMap

    Map generated through generateCandidateFileMap()

    Attributes
    protected
    Definition Classes
    DeltaCommand
  37. def hashCode(): Int
    Definition Classes
    TreeNode → AnyRef → Any
  38. def inferAdditionalConstraints(constraints: Set[Expression]): Set[Expression]
    Definition Classes
    ConstraintHelper
  39. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  40. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  41. def innerChildren: Seq[QueryPlan[_]]
    Definition Classes
    UpdateCommand → QueryPlan → TreeNode
  42. def inputSet: AttributeSet
    Definition Classes
    QueryPlan
  43. final def invalidateStatsCache(): Unit
    Definition Classes
    LogicalPlanStats
  44. def isCanonicalizedPlan: Boolean
    Attributes
    protected
    Definition Classes
    QueryPlan
  45. def isCatalogTable(analyzer: Analyzer, tableIdent: TableIdentifier): Boolean

    Use the analyzer to see whether the provided TableIdentifier is for a path based table or not

    Use the analyzer to see whether the provided TableIdentifier is for a path based table or not

    analyzer

    The session state analyzer to call

    tableIdent

    Table Identifier to determine whether is path based or not

    returns

    Boolean where true means that the table is a table in a metastore and false means the table is a path based table

    Definition Classes
    DeltaCommand
  46. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  47. def isPathIdentifier(tableIdent: TableIdentifier): Boolean

    Checks if the given identifier can be for a delta table's path

    Checks if the given identifier can be for a delta table's path

    tableIdent

    Table Identifier for which to check

    Attributes
    protected
    Definition Classes
    DeltaCommand
  48. def isStreaming: Boolean
    Definition Classes
    LogicalPlan
  49. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  50. def jsonFields: List[JField]
    Attributes
    protected
    Definition Classes
    TreeNode
  51. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  52. def logConsole(line: String): Unit
    Definition Classes
    DatabricksLogging
  53. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  54. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  55. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  56. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  57. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  58. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  59. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  60. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  61. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  62. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  63. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  64. def makeCopy(newArgs: Array[AnyRef]): LogicalPlan
    Definition Classes
    TreeNode
  65. def map[A](f: (LogicalPlan) ⇒ A): Seq[A]
    Definition Classes
    TreeNode
  66. def mapChildren(f: (LogicalPlan) ⇒ LogicalPlan): LogicalPlan
    Definition Classes
    TreeNode
  67. def mapExpressions(f: (Expression) ⇒ Expression): UpdateCommand.this.type
    Definition Classes
    QueryPlan
  68. def mapProductIterator[B](f: (Any) ⇒ B)(implicit arg0: ClassTag[B]): Array[B]
    Attributes
    protected
    Definition Classes
    TreeNode
  69. def maxRows: Option[Long]
    Definition Classes
    LogicalPlan
  70. def maxRowsPerPartition: Option[Long]
    Definition Classes
    LogicalPlan
  71. lazy val metrics: Map[String, SQLMetric]
    Definition Classes
    UpdateCommand → RunnableCommand
  72. def missingInput: AttributeSet
    Definition Classes
    QueryPlan
  73. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  74. def nodeName: String
    Definition Classes
    TreeNode
  75. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  76. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  77. def numberedTreeString: String
    Definition Classes
    TreeNode
  78. val origin: Origin
    Definition Classes
    TreeNode
  79. def otherCopyArgs: Seq[AnyRef]
    Attributes
    protected
    Definition Classes
    TreeNode
  80. def output: Seq[Attribute]
    Definition Classes
    Command → QueryPlan
  81. def outputOrdering: Seq[SortOrder]
    Definition Classes
    LogicalPlan
  82. def outputSet: AttributeSet
    Definition Classes
    QueryPlan
  83. def p(number: Int): LogicalPlan
    Definition Classes
    TreeNode
  84. def parsePartitionPredicates(spark: SparkSession, predicate: String): Seq[Expression]

    Converts string predicates into Expressions relative to a transaction.

    Converts string predicates into Expressions relative to a transaction.

    Attributes
    protected
    Definition Classes
    DeltaCommand
    Exceptions thrown

    AnalysisException if a non-partition column is referenced.

  85. def prettyJson: String
    Definition Classes
    TreeNode
  86. def printSchema(): Unit
    Definition Classes
    QueryPlan
  87. def producedAttributes: AttributeSet
    Definition Classes
    QueryPlan
  88. def recordDeltaEvent(deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty, data: AnyRef = null): Unit

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  89. def recordDeltaOperation[A](deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: ⇒ A): A

    Used to report the duration as well as the success or failure of an operation.

    Used to report the duration as well as the success or failure of an operation.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  90. def recordEvent(metric: MetricDefinition, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
    Definition Classes
    DatabricksLogging
  91. def recordOperation[S](opType: OpType, opTarget: String = null, extraTags: Map[TagDefinition, String], isSynchronous: Boolean = true, alwaysRecordStats: Boolean = false, allowAuthTags: Boolean = false, killJvmIfStuck: Boolean = false, outputMetric: MetricDefinition = null, silent: Boolean = true)(thunk: ⇒ S): S
    Definition Classes
    DatabricksLogging
  92. def recordUsage(metric: MetricDefinition, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
    Definition Classes
    DatabricksLogging
  93. def references: AttributeSet
    Definition Classes
    QueryPlan
  94. def refresh(): Unit
    Definition Classes
    LogicalPlan
  95. def removeFilesFromPaths(deltaLog: DeltaLog, nameToAddFileMap: Map[String, AddFile], filesToRewrite: Seq[String], operationTimestamp: Long): Seq[RemoveFile]

    This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.

    This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.

    deltaLog

    The DeltaLog of the table that is being operated on

    nameToAddFileMap

    A map generated using generateCandidateFileMap.

    filesToRewrite

    Absolute paths of the files that were touched. We will search for these in candidateFiles. Obtained as the output of the input_file_name function.

    operationTimestamp

    The timestamp of the operation

    Attributes
    protected
    Definition Classes
    DeltaCommand
  96. def resolve(nameParts: Seq[String], resolver: Resolver): Option[NamedExpression]
    Definition Classes
    LogicalPlan
  97. def resolve(schema: StructType, resolver: Resolver): Seq[Attribute]
    Definition Classes
    LogicalPlan
  98. def resolveChildren(nameParts: Seq[String], resolver: Resolver): Option[NamedExpression]
    Definition Classes
    LogicalPlan
  99. def resolveExpressions(r: PartialFunction[Expression, Expression]): LogicalPlan
    Definition Classes
    AnalysisHelper
  100. def resolveIdentifier(analyzer: Analyzer, identifier: TableIdentifier): LogicalPlan

    Use the analyzer to resolve the identifier provided

    Use the analyzer to resolve the identifier provided

    analyzer

    The session state analyzer to call

    identifier

    Table Identifier to determine whether is path based or not

    Attributes
    protected
    Definition Classes
    DeltaCommand
  101. def resolveOperators(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    AnalysisHelper
  102. def resolveOperatorsDown(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    AnalysisHelper
  103. def resolveOperatorsUp(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    AnalysisHelper
  104. def resolveQuoted(name: String, resolver: Resolver): Option[NamedExpression]
    Definition Classes
    LogicalPlan
  105. lazy val resolved: Boolean
    Definition Classes
    LogicalPlan
  106. final def run(sparkSession: SparkSession): Seq[Row]
    Definition Classes
    UpdateCommand → RunnableCommand
  107. final def sameResult(other: LogicalPlan): Boolean
    Definition Classes
    QueryPlan
  108. lazy val schema: StructType
    Definition Classes
    QueryPlan
  109. def schemaString: String
    Definition Classes
    QueryPlan
  110. final def semanticHash(): Int
    Definition Classes
    QueryPlan
  111. def simpleString: String
    Definition Classes
    QueryPlan → TreeNode
  112. def statePrefix: String
    Attributes
    protected
    Definition Classes
    LogicalPlan → QueryPlan
  113. def stats: Statistics
    Definition Classes
    LogicalPlanStats
  114. val statsCache: Option[Statistics]
    Attributes
    protected
    Definition Classes
    LogicalPlanStats
  115. def stringArgs: Iterator[Any]
    Attributes
    protected
    Definition Classes
    TreeNode
  116. def subqueries: Seq[LogicalPlan]
    Definition Classes
    QueryPlan
  117. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  118. val tahoeFileIndex: TahoeFileIndex
  119. val target: LogicalPlan
  120. def toJSON: String
    Definition Classes
    TreeNode
  121. def toString(): String
    Definition Classes
    TreeNode → AnyRef → Any
  122. def transform(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    TreeNode
  123. def transformAllExpressions(rule: PartialFunction[Expression, Expression]): UpdateCommand.this.type
    Definition Classes
    AnalysisHelper → QueryPlan
  124. def transformDown(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    AnalysisHelper → TreeNode
  125. def transformExpressions(rule: PartialFunction[Expression, Expression]): UpdateCommand.this.type
    Definition Classes
    QueryPlan
  126. def transformExpressionsDown(rule: PartialFunction[Expression, Expression]): UpdateCommand.this.type
    Definition Classes
    QueryPlan
  127. def transformExpressionsUp(rule: PartialFunction[Expression, Expression]): UpdateCommand.this.type
    Definition Classes
    QueryPlan
  128. def transformUp(rule: PartialFunction[LogicalPlan, LogicalPlan]): LogicalPlan
    Definition Classes
    AnalysisHelper → TreeNode
  129. def treeString(verbose: Boolean, addSuffix: Boolean): String
    Definition Classes
    TreeNode
  130. def treeString: String
    Definition Classes
    TreeNode
  131. val updateExpressions: Seq[Expression]
  132. def validConstraints: Set[Expression]
    Attributes
    protected
    Definition Classes
    QueryPlanConstraints
  133. def verboseString: String
    Definition Classes
    QueryPlan → TreeNode
  134. def verboseStringWithSuffix: String
    Definition Classes
    LogicalPlan → TreeNode
  135. def verifyPartitionPredicates(spark: SparkSession, partitionColumns: Seq[String], predicates: Seq[Expression]): Unit
    Attributes
    protected
    Definition Classes
    DeltaCommand
  136. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  137. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  138. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  139. def withNewChildren(newChildren: Seq[LogicalPlan]): LogicalPlan
    Definition Classes
    TreeNode
  140. def withStatusCode[T](statusCode: String, defaultMessage: String, data: Map[String, Any] = Map.empty)(body: ⇒ T): T

    Report a log to indicate some command is running.

    Report a log to indicate some command is running.

    Definition Classes
    DeltaProgressReporter

Inherited from Serializable

Inherited from Serializable

Inherited from DeltaCommand

Inherited from DeltaLogging

Inherited from DatabricksLogging

Inherited from DeltaProgressReporter

Inherited from RunnableCommand

Inherited from Command

Inherited from LogicalPlan

Inherited from Logging

Inherited from QueryPlanConstraints

Inherited from ConstraintHelper

Inherited from LogicalPlanStats

Inherited from AnalysisHelper

Inherited from QueryPlan[LogicalPlan]

Inherited from TreeNode[LogicalPlan]

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped