catalog

Type Members

1. case class BucketSpec(numBuckets: Int, bucketColumnNames: Seq[String], sortColumnNames: Seq[String]) extends Product with Serializable

   A container for bucketing information.

2. case class CatalogDatabase(name: String, description: String, locationUri: URI, properties: Map[String, String]) extends Product with Serializable

   A database defined in the catalog.

3. case class CatalogFunction(identifier: FunctionIdentifier, className: String, resources: Seq[FunctionResource]) extends Product with Serializable

   A function defined in the catalog.

4. case class CatalogRelation(tableMeta: CatalogTable, dataCols: Seq[AttributeReference], partitionCols: Seq[AttributeReference]) extends LeafNode with MultiInstanceRelation with Product with Serializable

   A LogicalPlan that represents a table.

5. case class CatalogStatistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = scala.None, colStats: Map[String, ColumnStat] = ...) extends Product with Serializable

   This class of statistics is used in CatalogTable to interact with metastore.

6. case class CatalogStorageFormat(locationUri: Option[URI], inputFormat: Option[String], outputFormat: Option[String], serde: Option[String], compressed: Boolean, properties: Map[String, String]) extends Product with Serializable

   Storage format, used to describe how a partition or a table is stored.

7. case class CatalogTable(identifier: TableIdentifier, tableType: CatalogTableType, storage: CatalogStorageFormat, schema: StructType, provider: Option[String] = scala.None, partitionColumnNames: Seq[String] = collection.this.Seq.empty[Nothing], bucketSpec: Option[BucketSpec] = scala.None, owner: String = "", createTime: Long = ..., lastAccessTime: Long = -1, properties: Map[String, String] = ..., stats: Option[CatalogStatistics] = scala.None, viewText: Option[String] = scala.None, comment: Option[String] = scala.None, unsupportedFeatures: Seq[String] = collection.this.Seq.empty[Nothing], tracksPartitionsInCatalog: Boolean = false, schemaPreservesCase: Boolean = true) extends Product with Serializable

   A table defined in the catalog.

8. case class CatalogTablePartition(spec: TablePartitionSpec, storage: CatalogStorageFormat, parameters: Map[String, String] = ...) extends Product with Serializable

   A partition (Hive style) defined in the catalog.

9. case class CatalogTableType extends Product with Serializable

10. case class CreateDatabaseEvent(database: String) extends DatabaseEvent with Product with Serializable

    Event fired after a database has been created.

11. case class CreateDatabasePreEvent(database: String) extends DatabaseEvent with Product with Serializable

    Event fired before a database is created.

12. case class CreateFunctionEvent(database: String, name: String) extends FunctionEvent with Product with Serializable

    Event fired after a function has been created.

13. case class CreateFunctionPreEvent(database: String, name: String) extends FunctionEvent with Product with Serializable

    Event fired before a function is created.

14. case class CreateTableEvent(database: String, name: String) extends TableEvent with Product with Serializable

    Event fired after a table has been created.

15. case class CreateTablePreEvent(database: String, name: String) extends TableEvent with Product with Serializable

    Event fired before a table is created.

16. trait DatabaseEvent extends ExternalCatalogEvent

    Event fired when a database is create or dropped.

17. case class DropDatabaseEvent(database: String) extends DatabaseEvent with Product with Serializable

    Event fired after a database has been dropped.

18. case class DropDatabasePreEvent(database: String) extends DatabaseEvent with Product with Serializable

    Event fired before a database is dropped.

19. case class DropFunctionEvent(database: String, name: String) extends FunctionEvent with Product with Serializable

    Event fired after a function has been dropped.

20. case class DropFunctionPreEvent(database: String, name: String) extends FunctionEvent with Product with Serializable

    Event fired before a function is dropped.

21. case class DropTableEvent(database: String, name: String) extends TableEvent with Product with Serializable

    Event fired after a table has been dropped.

22. case class DropTablePreEvent(database: String, name: String) extends TableEvent with Product with Serializable

    Event fired before a table is dropped.

23. abstract class ExternalCatalog extends ListenerBus[ExternalCatalogEventListener, ExternalCatalogEvent]

    Interface for the system catalog (of functions, partitions, tables, and databases).

24. trait ExternalCatalogEvent extends SparkListenerEvent

    Event emitted by the external catalog when it is modified.

25. trait ExternalCatalogEventListener extends AnyRef

    Listener interface for external catalog modification events.

26. trait FunctionEvent extends DatabaseEvent

    Event fired when a function is created, dropped or renamed.

27. case class FunctionResource(resourceType: FunctionResourceType, uri: String) extends Product with Serializable

28. trait FunctionResourceLoader extends AnyRef

    A simple trait representing a class that can be used to load resources used by a function.

29. abstract class FunctionResourceType extends AnyRef

    A trait that represents the type of a resourced needed by a function.

30. class GlobalTempViewManager extends AnyRef

    A thread-safe manager for global temporary views, providing atomic operations to manage them, e.

31. class InMemoryCatalog extends ExternalCatalog

    An in-memory (ephemeral) implementation of the system catalog.

32. case class RenameFunctionEvent(database: String, name: String, newName: String) extends FunctionEvent with Product with Serializable

    Event fired after a function has been renamed.

33. case class RenameFunctionPreEvent(database: String, name: String, newName: String) extends FunctionEvent with Product with Serializable

    Event fired before a function is renamed.

34. case class RenameTableEvent(database: String, name: String, newName: String) extends TableEvent with Product with Serializable

    Event fired after a table has been renamed.

35. case class RenameTablePreEvent(database: String, name: String, newName: String) extends TableEvent with Product with Serializable

    Event fired before a table is renamed.

36. class SessionCatalog extends Logging

    An internal catalog that is used by a Spark Session.

37. trait TableEvent extends DatabaseEvent

    Event fired when a table is created, dropped or renamed.