class DeltaCatalog extends AnyRef
Delta Catalog implementation. This class executes calls to _delta_log for catalog operations such as createTable, getTable etc. This class also prepares, persists and uses data store in metastore using decorated catalog implementation.
Catalog operations that are not in scope of Delta Table or do not require _delta_log operations
will be handled by CatalogProxy and BaseCatalog classes.
- Alphabetic
- By Inheritance
- DeltaCatalog
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
alterTable(newCatalogTable: DeltaCatalogBaseTable): Unit
Executes ALTER operation on Delta table.
Executes ALTER operation on Delta table. Currently, only changing table name and changing/setting table properties is supported using ALTER statement.
Changing table name:
ALTER TABLE sourceTable RENAME TO newSourceTableSetting table property:
ALTER TABLE sourceTable SET ('userCustomProp'='myVal')- newCatalogTable
catalog table with new name and properties defined by ALTER statement.
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
createTable(catalogTable: DeltaCatalogBaseTable, ignoreIfExists: Boolean): Unit
Creates a new table in metastore and _delta_log if not already exists under given table Path.
Creates a new table in metastore and _delta_log if not already exists under given table Path. The information stored in metastore will contain only catalog path (database.tableName) and connector type. DDL options and table schema will be stored in _delta_log.
If _delta_log already exists under DDL's table-path option this method will throw an exception if DDL scheme does not match _delta_log schema or DDL options override existing _delta_log table properties or Partition specification defined in
PARTITION BYdoes not match _delta_log partition specification.The framework will make sure to call this method with fully validated ResolvedCatalogTable or ResolvedCatalogView.
- catalogTable
the
DeltaCatalogBaseTablewith describing new table that should be added to the catalog.- ignoreIfExists
specifies behavior when a table or view already exists at the given path: if set to false, it throws a TableAlreadyExistException, if set to true, do nothing.
- Exceptions thrown
CatalogExceptionin case of any runtime exceptionDatabaseNotExistExceptionif the database in tablePath doesn't existTableAlreadyExistExceptionif table already exists and ignoreIfExists is false
-
def
dropTable(catalogTable: DeltaCatalogBaseTable, ignoreIfExists: Boolean): Unit
Deletes metastore entry and clears DeltaCatalog cache for given Delta table.
Deletes metastore entry and clears DeltaCatalog cache for given Delta table.
By design, we will remove only metastore information during drop table. No filesystem information (for example _delta_log folder) will be removed. However, we have to clear DeltaCatalog's cache for this table.
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[java.lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getTable(catalogTable: DeltaCatalogBaseTable): CatalogBaseTable
Returns a
CatalogBaseTableidentified by the givenDeltaCatalogBaseTable#getCatalogTable().Returns a
CatalogBaseTableidentified by the givenDeltaCatalogBaseTable#getCatalogTable(). This method assumes that providedDeltaCatalogBaseTable#getCatalogTable()table already exists in metastore hence no extra metastore checks will be executed.- Exceptions thrown
TableNotExistExceptionif the target does not exist
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
tableExists(catalogTable: DeltaCatalogBaseTable): Boolean
Checks if _delta_log folder exists for table described by
DeltaCatalogBaseTable#getCatalogTable()metastore entry.Checks if _delta_log folder exists for table described by
DeltaCatalogBaseTable#getCatalogTable()metastore entry. This method assumes that table exists in metastore thus not execute any checks there.- returns
true if _delta_log exists for given
DeltaCatalogBaseTable, false if not.
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()