Value Members

1.  final def !=(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

2.  final def ##(): Int

   Definition Classes

   AnyRef → Any

3.  final def ==(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

4.  final def asInstanceOf[T0]: T0

   Definition Classes

   Any

5.  def buildBaseRelation(spark: SparkSession, txn: OptimisticTransaction, actionType: String, rootPath: Path, inputLeafFiles: Seq[String], nameToAddFileMap: Map[String, AddFile]): HadoopFsRelation

   Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.

   Build a base relation of files that need to be rewritten as part of an update/delete/merge operation.

   Attributes

   protected

6.  def clone(): AnyRef

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( ... ) @native()

7.  final def eq(arg0: AnyRef): Boolean

   Definition Classes

   AnyRef

8.  def equals(arg0: Any): Boolean

   Definition Classes

   AnyRef → Any

9.  def finalize(): Unit

   Attributes

   protected[lang]

   Definition Classes

   AnyRef

   Annotations

   @throws( classOf[java.lang.Throwable] )

10.  def generateCandidateFileMap(basePath: Path, candidateFiles: Seq[AddFile]): Map[String, AddFile]

    Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update.

    Generates a map of file names to add file entries for operations where we will need to rewrite files such as delete, merge, update. We expect file names to be unique, because each file contains a UUID.

    Attributes

    protected

11.  final def getClass(): Class[_]

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

12.  def getTouchedFile(basePath: Path, filePath: String, nameToAddFileMap: Map[String, AddFile]): AddFile

    Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.

    Find the AddFile record corresponding to the file that was read as part of a delete/update/merge operation.

    filePath

    The path to a file. Can be either absolute or relative

    nameToAddFileMap

    Map generated through generateCandidateFileMap()

    Attributes

    protected

13.  def hashCode(): Int

    Definition Classes

    AnyRef → Any

    Annotations

    @native()

14.  def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

    Attributes

    protected

    Definition Classes

    Logging

15.  def initializeLogIfNecessary(isInterpreter: Boolean): Unit

    Attributes

    protected

    Definition Classes

    Logging

16.  def isCatalogTable(analyzer: Analyzer, tableIdent: TableIdentifier): Boolean

    Use the analyzer to see whether the provided TableIdentifier is for a path based table or not

    Use the analyzer to see whether the provided TableIdentifier is for a path based table or not

    analyzer

    The session state analyzer to call

    tableIdent

    Table Identifier to determine whether is path based or not

    returns

    Boolean where true means that the table is a table in a metastore and false means the table is a path based table

17.  final def isInstanceOf[T0]: Boolean

    Definition Classes

    Any

18.  def isPathIdentifier(tableIdent: TableIdentifier): Boolean

    Checks if the given identifier can be for a delta table's path

    Checks if the given identifier can be for a delta table's path

    tableIdent

    Table Identifier for which to check

    Attributes

    protected

19.  def isTraceEnabled(): Boolean

    Attributes

    protected

    Definition Classes

    Logging

20.  def log: Logger

    Attributes

    protected

    Definition Classes

    Logging

21.  def logConsole(line: String): Unit

    Definition Classes

    DatabricksLogging

22.  def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

23.  def logDebug(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

24.  def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

25.  def logError(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

26.  def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

27.  def logInfo(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

28.  def logName: String

    Attributes

    protected

    Definition Classes

    Logging

29.  def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

30.  def logTrace(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

31.  def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes

    protected

    Definition Classes

    Logging

32.  def logWarning(msg: ⇒ String): Unit

    Attributes

    protected

    Definition Classes

    Logging

33.  final def ne(arg0: AnyRef): Boolean

    Definition Classes

    AnyRef

34.  final def notify(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

35.  final def notifyAll(): Unit

    Definition Classes

    AnyRef

    Annotations

    @native()

36.  def parsePartitionPredicates(spark: SparkSession, predicate: String): Seq[Expression]

    Converts string predicates into Expressions relative to a transaction.

    Converts string predicates into Expressions relative to a transaction.

    Attributes

    protected

    Exceptions thrown

    AnalysisException if a non-partition column is referenced.

37.  def recordDeltaEvent(deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty, data: AnyRef = null): Unit

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    Attributes

    protected

    Definition Classes

    DeltaLogging

38.  def recordDeltaOperation[A](deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: ⇒ A): A

    Used to report the duration as well as the success or failure of an operation.

    Used to report the duration as well as the success or failure of an operation.

    Attributes

    protected

    Definition Classes

    DeltaLogging

39.  def recordEvent(metric: MetricDefinition, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit

    Definition Classes

    DatabricksLogging

40.  def recordOperation[S](opType: OpType, opTarget: String = null, extraTags: Map[TagDefinition, String], isSynchronous: Boolean = true, alwaysRecordStats: Boolean = false, allowAuthTags: Boolean = false, killJvmIfStuck: Boolean = false, outputMetric: MetricDefinition = null, silent: Boolean = true)(thunk: ⇒ S): S

    Definition Classes

    DatabricksLogging

41.  def recordUsage(metric: MetricDefinition, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit

    Definition Classes

    DatabricksLogging

42.  def removeFilesFromPaths(deltaLog: DeltaLog, nameToAddFileMap: Map[String, AddFile], filesToRewrite: Seq[String], operationTimestamp: Long): Seq[RemoveFile]

    This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.

    This method provides the RemoveFile actions that are necessary for files that are touched and need to be rewritten in methods like Delete, Update, and Merge.

    deltaLog

    The DeltaLog of the table that is being operated on

    nameToAddFileMap

    A map generated using generateCandidateFileMap.

    filesToRewrite

    Absolute paths of the files that were touched. We will search for these in candidateFiles. Obtained as the output of the input_file_name function.

    operationTimestamp

    The timestamp of the operation

    Attributes

    protected

43.  def resolveIdentifier(analyzer: Analyzer, identifier: TableIdentifier): LogicalPlan

    Use the analyzer to resolve the identifier provided

    Use the analyzer to resolve the identifier provided

    analyzer

    The session state analyzer to call

    identifier

    Table Identifier to determine whether is path based or not

    Attributes

    protected

44.  final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes

    AnyRef

45.  def toString(): String

    Definition Classes

    AnyRef → Any

46.  def verifyPartitionPredicates(spark: SparkSession, partitionColumns: Seq[String], predicates: Seq[Expression]): Unit

    Attributes

    protected

47.  final def wait(): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

48.  final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... )

49.  final def wait(arg0: Long): Unit

    Definition Classes

    AnyRef

    Annotations

    @throws( ... ) @native()

50.  def withStatusCode[T](statusCode: String, defaultMessage: String, data: Map[String, Any] = Map.empty)(body: ⇒ T): T

    Report a log to indicate some command is running.

    Report a log to indicate some command is running.

    Definition Classes

    DeltaProgressReporter