Packages

class DeltaCatalog extends DelegatingCatalogExtension with StagingTableCatalog with SupportsPathIdentifier with DeltaLogging

A Catalog extension which can properly handle the interaction between the HiveMetaStore and Delta tables. It delegates all operations DataSources other than Delta to the SparkCatalog.

Linear Supertypes
DeltaLogging, DatabricksLogging, DeltaProgressReporter, LoggingShims, Logging, SupportsPathIdentifier, StagingTableCatalog, DelegatingCatalogExtension, CatalogExtension, SupportsNamespaces, FunctionCatalog, TableCatalog, CatalogPlugin, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. DeltaCatalog
  2. DeltaLogging
  3. DatabricksLogging
  4. DeltaProgressReporter
  5. LoggingShims
  6. Logging
  7. SupportsPathIdentifier
  8. StagingTableCatalog
  9. DelegatingCatalogExtension
  10. CatalogExtension
  11. SupportsNamespaces
  12. FunctionCatalog
  13. TableCatalog
  14. CatalogPlugin
  15. AnyRef
  16. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new DeltaCatalog()

Type Members

  1. implicit class LogStringContext extends AnyRef
    Definition Classes
    LoggingShims

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def alterNamespace(arg0: Array[String], arg1: <repeated...>[NamespaceChange]): Unit
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... ) @transient()
  5. def alterTable(ident: Identifier, changes: TableChange*): Table
  6. def alterTable(arg0: Identifier, arg1: <repeated...>[TableChange]): Table
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
    Annotations
    @throws( ... ) @transient()
  7. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  8. def capabilities(): Set[TableCatalogCapability]
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
  9. lazy val catalog: SessionCatalog
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  10. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  11. def createNamespace(arg0: Array[String], arg1: Map[String, String]): Unit
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... )
  12. def createTable(ident: Identifier, schema: StructType, partitions: Array[Transform], properties: Map[String, String]): Table
    Definition Classes
    DeltaCatalog → DelegatingCatalogExtension → TableCatalog
  13. def createTable(ident: Identifier, columns: Array[connector.catalog.Column], partitions: Array[Transform], properties: Map[String, String]): Table
    Definition Classes
    DeltaCatalog → DelegatingCatalogExtension → TableCatalog
  14. def defaultNamespace(): Array[String]
    Definition Classes
    DelegatingCatalogExtension → CatalogPlugin
  15. def deltaAssert(check: ⇒ Boolean, name: String, msg: String, deltaLog: DeltaLog = null, data: AnyRef = null, path: Option[Path] = None): Unit

    Helper method to check invariants in Delta code.

    Helper method to check invariants in Delta code. Fails when running in tests, records a delta assertion event and logs a warning otherwise.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  16. def dropNamespace(arg0: Array[String], arg1: Boolean): Boolean
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... ) @throws( ... )
  17. def dropTable(arg0: Identifier): Boolean
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
  18. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  19. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  20. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  21. def functionExists(arg0: Identifier): Boolean
    Definition Classes
    DelegatingCatalogExtension → FunctionCatalog
  22. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  23. def getCommonTags(deltaLog: DeltaLog, tahoeId: String): Map[TagDefinition, String]
    Definition Classes
    DeltaLogging
  24. def getErrorData(e: Throwable): Map[String, Any]
    Definition Classes
    DeltaLogging
  25. def getExistingTableIfExists(table: TableIdentifier): Option[CatalogTable]

    Checks if a table already exists for the provided identifier.

  26. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  27. final def initialize(arg0: String, arg1: CaseInsensitiveStringMap): Unit
    Definition Classes
    DelegatingCatalogExtension → CatalogPlugin
  28. def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  29. def initializeLogIfNecessary(isInterpreter: Boolean): Unit
    Attributes
    protected
    Definition Classes
    Logging
  30. def invalidateTable(arg0: Identifier): Unit
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
  31. def isIcebergPathIdentifier(ident: Identifier): Boolean
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  32. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  33. def isPathIdentifier(tableIdentifier: TableIdentifier): Boolean
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  34. def isPathIdentifier(table: CatalogTable): Boolean
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  35. def isPathIdentifier(ident: Identifier): Boolean
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  36. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  37. def listFunctions(arg0: Array[String]): Array[Identifier]
    Definition Classes
    DelegatingCatalogExtension → FunctionCatalog
    Annotations
    @throws( ... )
  38. def listNamespaces(arg0: Array[String]): Array[Array[String]]
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... )
  39. def listNamespaces(): Array[Array[String]]
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... )
  40. def listTables(arg0: Array[String]): Array[Identifier]
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
    Annotations
    @throws( ... )
  41. def loadFunction(arg0: Identifier): UnboundFunction
    Definition Classes
    DelegatingCatalogExtension → FunctionCatalog
    Annotations
    @throws( ... )
  42. def loadNamespaceMetadata(arg0: Array[String]): Map[String, String]
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
    Annotations
    @throws( ... )
  43. def loadTable(ident: Identifier, version: String): Table
    Definition Classes
    DeltaCatalog → DelegatingCatalogExtension → TableCatalog
  44. def loadTable(ident: Identifier, timestamp: Long): Table
    Definition Classes
    DeltaCatalog → DelegatingCatalogExtension → TableCatalog
  45. def loadTable(ident: Identifier): Table
    Definition Classes
    DeltaCatalog → DelegatingCatalogExtension → TableCatalog
  46. def loadTable(arg0: Identifier, arg1: Set[TableWritePrivilege]): Table
    Definition Classes
    TableCatalog
    Annotations
    @throws( ... )
  47. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  48. def logConsole(line: String): Unit
    Definition Classes
    DatabricksLogging
  49. def logDebug(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  50. def logDebug(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  51. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  52. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  53. def logError(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  54. def logError(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  55. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  56. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  57. def logInfo(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  58. def logInfo(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  59. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  60. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  61. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  62. def logTrace(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  63. def logTrace(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  64. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  65. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  66. def logWarning(entry: LogEntry, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  67. def logWarning(entry: LogEntry): Unit
    Attributes
    protected
    Definition Classes
    LoggingShims
  68. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  69. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  70. def name(): String
    Definition Classes
    DelegatingCatalogExtension → CatalogPlugin
  71. def namespaceExists(arg0: Array[String]): Boolean
    Definition Classes
    DelegatingCatalogExtension → SupportsNamespaces
  72. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  73. def newDeltaPathTable(ident: Identifier): DeltaTableV2
    Attributes
    protected
  74. def newIcebergPathTable(ident: Identifier): IcebergTablePlaceHolder
    Attributes
    protected
    Definition Classes
    SupportsPathIdentifier
  75. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  76. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  77. def purgeTable(arg0: Identifier): Boolean
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
  78. def recordDeltaEvent(deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty, data: AnyRef = null, path: Option[Path] = None): Unit

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    Used to record the occurrence of a single event or report detailed, operation specific statistics.

    path

    Used to log the path of the delta table when deltaLog is null.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  79. def recordDeltaOperation[A](deltaLog: DeltaLog, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: ⇒ A): A

    Used to report the duration as well as the success or failure of an operation on a deltaLog.

    Used to report the duration as well as the success or failure of an operation on a deltaLog.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  80. def recordDeltaOperationForTablePath[A](tablePath: String, opType: String, tags: Map[TagDefinition, String] = Map.empty)(thunk: ⇒ A): A

    Used to report the duration as well as the success or failure of an operation on a tahoePath.

    Used to report the duration as well as the success or failure of an operation on a tahoePath.

    Attributes
    protected
    Definition Classes
    DeltaLogging
  81. def recordEvent(metric: MetricDefinition, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
    Definition Classes
    DatabricksLogging
  82. def recordFrameProfile[T](group: String, name: String)(thunk: ⇒ T): T
    Attributes
    protected
    Definition Classes
    DeltaLogging
  83. def recordOperation[S](opType: OpType, opTarget: String = null, extraTags: Map[TagDefinition, String], isSynchronous: Boolean = true, alwaysRecordStats: Boolean = false, allowAuthTags: Boolean = false, killJvmIfStuck: Boolean = false, outputMetric: MetricDefinition = METRIC_OPERATION_DURATION, silent: Boolean = true)(thunk: ⇒ S): S
    Definition Classes
    DatabricksLogging
  84. def recordProductEvent(metric: MetricDefinition with CentralizableMetric, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, trimBlob: Boolean = true): Unit
    Definition Classes
    DatabricksLogging
  85. def recordProductUsage(metric: MetricDefinition with CentralizableMetric, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
    Definition Classes
    DatabricksLogging
  86. def recordUsage(metric: MetricDefinition, quantity: Double, additionalTags: Map[TagDefinition, String] = Map.empty, blob: String = null, forceSample: Boolean = false, trimBlob: Boolean = true, silent: Boolean = false): Unit
    Definition Classes
    DatabricksLogging
  87. def renameTable(arg0: Identifier, arg1: Identifier): Unit
    Definition Classes
    DelegatingCatalogExtension → TableCatalog
    Annotations
    @throws( ... ) @throws( ... )
  88. final def setDelegateCatalog(arg0: CatalogPlugin): Unit
    Definition Classes
    DelegatingCatalogExtension → CatalogExtension
  89. val spark: SparkSession
  90. def stageCreate(ident: Identifier, schema: StructType, partitions: Array[Transform], properties: Map[String, String]): StagedTable
    Definition Classes
    DeltaCatalog → StagingTableCatalog
  91. def stageCreate(arg0: Identifier, arg1: Array[connector.catalog.Column], arg2: Array[Transform], arg3: Map[String, String]): StagedTable
    Definition Classes
    StagingTableCatalog
    Annotations
    @throws( ... ) @throws( ... )
  92. def stageCreateOrReplace(ident: Identifier, schema: StructType, partitions: Array[Transform], properties: Map[String, String]): StagedTable
    Definition Classes
    DeltaCatalog → StagingTableCatalog
  93. def stageCreateOrReplace(arg0: Identifier, arg1: Array[connector.catalog.Column], arg2: Array[Transform], arg3: Map[String, String]): StagedTable
    Definition Classes
    StagingTableCatalog
    Annotations
    @throws( ... )
  94. def stageReplace(ident: Identifier, schema: StructType, partitions: Array[Transform], properties: Map[String, String]): StagedTable
    Definition Classes
    DeltaCatalog → StagingTableCatalog
  95. def stageReplace(arg0: Identifier, arg1: Array[connector.catalog.Column], arg2: Array[Transform], arg3: Map[String, String]): StagedTable
    Definition Classes
    StagingTableCatalog
    Annotations
    @throws( ... ) @throws( ... )
  96. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  97. def tableExists(ident: Identifier): Boolean
    Definition Classes
    SupportsPathIdentifier → TableCatalog
  98. def toString(): String
    Definition Classes
    AnyRef → Any
  99. def useNullableQuerySchema(): Boolean
    Definition Classes
    TableCatalog
  100. def validateClusterBySpec(maybeClusterBySpec: Option[ClusterBySpec], schema: StructType): Unit
  101. def verifyTableAndSolidify(tableDesc: CatalogTable, query: Option[LogicalPlan], maybeClusterBySpec: Option[ClusterBySpec] = None): CatalogTable

    Performs checks on the parameters provided for table creation for a Delta table.

  102. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  103. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  104. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  105. def withStatusCode[T](statusCode: String, defaultMessage: String, data: Map[String, Any] = Map.empty)(body: ⇒ T): T

    Report a log to indicate some command is running.

    Report a log to indicate some command is running.

    Definition Classes
    DeltaProgressReporter

Inherited from DeltaLogging

Inherited from DatabricksLogging

Inherited from DeltaProgressReporter

Inherited from LoggingShims

Inherited from Logging

Inherited from SupportsPathIdentifier

Inherited from StagingTableCatalog

Inherited from DelegatingCatalogExtension

Inherited from CatalogExtension

Inherited from SupportsNamespaces

Inherited from FunctionCatalog

Inherited from TableCatalog

Inherited from CatalogPlugin

Inherited from AnyRef

Inherited from Any

Ungrouped