Class

org.apache.spark.sql.catalyst.catalog

InMemoryCatalog

Related Doc: package catalog

Permalink

class InMemoryCatalog extends ExternalCatalog

An in-memory (ephemeral) implementation of the system catalog.

This is a dummy implementation that does not require setting up external systems. It is intended for testing or exploration purposes only and should not be used in production.

All public methods should be synchronized for thread-safety.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. InMemoryCatalog
  2. ExternalCatalog
  3. AnyRef
  4. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new InMemoryCatalog(conf: SparkConf = new SparkConf, hadoopConfig: Configuration = new Configuration)

    Permalink

Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  4. def alterDatabase(dbDefinition: CatalogDatabase): Unit

    Permalink

    Alter a database whose name matches the one specified in dbDefinition, assuming the database exists.

    Alter a database whose name matches the one specified in dbDefinition, assuming the database exists.

    Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.

    Definition Classes
    InMemoryCatalogExternalCatalog
  5. def alterPartitions(db: String, table: String, parts: Seq[CatalogTablePartition]): Unit

    Permalink

    Alter one or many table partitions whose specs that match those specified in parts, assuming the partitions exist.

    Alter one or many table partitions whose specs that match those specified in parts, assuming the partitions exist.

    Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.

    Definition Classes
    InMemoryCatalogExternalCatalog
  6. def alterTable(tableDefinition: CatalogTable): Unit

    Permalink

    Alter a table whose database and name match the ones specified in tableDefinition, assuming the table exists.

    Alter a table whose database and name match the ones specified in tableDefinition, assuming the table exists. Note that, even though we can specify database in tableDefinition, it's used to identify the table, not to alter the table's database, which is not allowed.

    Note: If the underlying implementation does not support altering a certain field, this becomes a no-op.

    Definition Classes
    InMemoryCatalogExternalCatalog
  7. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any
  8. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  9. def createDatabase(dbDefinition: CatalogDatabase, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  10. def createFunction(db: String, func: CatalogFunction): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  11. def createPartitions(db: String, table: String, parts: Seq[CatalogTablePartition], ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  12. def createTable(tableDefinition: CatalogTable, ignoreIfExists: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  13. def databaseExists(db: String): Boolean

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  14. def dropDatabase(db: String, ignoreIfNotExists: Boolean, cascade: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  15. def dropFunction(db: String, funcName: String): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  16. def dropPartitions(db: String, table: String, partSpecs: Seq[TablePartitionSpec], ignoreIfNotExists: Boolean, purge: Boolean, retainData: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  17. def dropTable(db: String, table: String, ignoreIfNotExists: Boolean, purge: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  18. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  19. def equals(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any
  20. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  21. def functionExists(db: String, funcName: String): Boolean

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  22. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any
  23. def getDatabase(db: String): CatalogDatabase

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  24. def getFunction(db: String, funcName: String): CatalogFunction

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  25. def getPartition(db: String, table: String, spec: TablePartitionSpec): CatalogTablePartition

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  26. def getPartitionOption(db: String, table: String, spec: TablePartitionSpec): Option[CatalogTablePartition]

    Permalink

    Returns the specified partition or None if it does not exist.

    Returns the specified partition or None if it does not exist.

    Definition Classes
    InMemoryCatalogExternalCatalog
  27. def getTable(db: String, table: String): CatalogTable

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  28. def getTableOption(db: String, table: String): Option[CatalogTable]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  29. def hashCode(): Int

    Permalink
    Definition Classes
    AnyRef → Any
  30. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any
  31. def listDatabases(pattern: String): Seq[String]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  32. def listDatabases(): Seq[String]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  33. def listFunctions(db: String, pattern: String): Seq[String]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  34. def listPartitionNames(db: String, table: String, partialSpec: Option[TablePartitionSpec] = None): Seq[String]

    Permalink

    List the names of all partitions that belong to the specified table, assuming it exists.

    List the names of all partitions that belong to the specified table, assuming it exists.

    For a table with partition columns p1, p2, p3, each partition name is formatted as p1=v1/p2=v2/p3=v3. Each partition column name and value is an escaped path name, and can be decoded with the ExternalCatalogUtils.unescapePathName method.

    The returned sequence is sorted as strings.

    A partial partition spec may optionally be provided to filter the partitions returned, as described in the listPartitions method.

    db

    database name

    table

    table name

    partialSpec

    partition spec

    Definition Classes
    InMemoryCatalogExternalCatalog
  35. def listPartitions(db: String, table: String, partialSpec: Option[TablePartitionSpec] = None): Seq[CatalogTablePartition]

    Permalink

    List the metadata of all partitions that belong to the specified table, assuming it exists.

    List the metadata of all partitions that belong to the specified table, assuming it exists.

    A partial partition spec may optionally be provided to filter the partitions returned. For instance, if there exist partitions (a='1', b='2'), (a='1', b='3') and (a='2', b='4'), then a partial spec of (a='1') will return the first two only.

    db

    database name

    table

    table name

    partialSpec

    partition spec

    Definition Classes
    InMemoryCatalogExternalCatalog
  36. def listPartitionsByFilter(db: String, table: String, predicates: Seq[Expression]): Seq[CatalogTablePartition]

    Permalink

    List the metadata of partitions that belong to the specified table, assuming it exists, that satisfy the given partition-pruning predicate expressions.

    List the metadata of partitions that belong to the specified table, assuming it exists, that satisfy the given partition-pruning predicate expressions.

    db

    database name

    table

    table name

    predicates

    partition-pruning predicates

    Definition Classes
    InMemoryCatalogExternalCatalog
  37. def listTables(db: String, pattern: String): Seq[String]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  38. def listTables(db: String): Seq[String]

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  39. def loadDynamicPartitions(db: String, table: String, loadPath: String, partition: TablePartitionSpec, replace: Boolean, numDP: Int, holdDDLTime: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  40. def loadPartition(db: String, table: String, loadPath: String, partition: TablePartitionSpec, isOverwrite: Boolean, holdDDLTime: Boolean, inheritTableSpecs: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  41. def loadTable(db: String, table: String, loadPath: String, isOverwrite: Boolean, holdDDLTime: Boolean): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  42. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef
  43. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef
  44. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef
  45. def renameFunction(db: String, oldName: String, newName: String): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  46. def renamePartitions(db: String, table: String, specs: Seq[TablePartitionSpec], newSpecs: Seq[TablePartitionSpec]): Unit

    Permalink

    Override the specs of one or many existing table partitions, assuming they exist.

    Override the specs of one or many existing table partitions, assuming they exist. This assumes index i of specs corresponds to index i of newSpecs.

    Definition Classes
    InMemoryCatalogExternalCatalog
  47. def renameTable(db: String, oldName: String, newName: String): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  48. def requireDbExists(db: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    ExternalCatalog
  49. def requireFunctionExists(db: String, funcName: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    ExternalCatalog
  50. def requireFunctionNotExists(db: String, funcName: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    ExternalCatalog
  51. def requireTableExists(db: String, table: String): Unit

    Permalink
    Attributes
    protected
    Definition Classes
    ExternalCatalog
  52. def setCurrentDatabase(db: String): Unit

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  53. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef
  54. def tableExists(db: String, table: String): Boolean

    Permalink
    Definition Classes
    InMemoryCatalogExternalCatalog
  55. def toString(): String

    Permalink
    Definition Classes
    AnyRef → Any
  56. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  57. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  58. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from ExternalCatalog

Inherited from AnyRef

Inherited from Any

Ungrouped