org.apache.spark.sql.catalyst.catalog

CatalogTable

case class CatalogTable(identifier: TableIdentifier, tableType: CatalogTableType, storage: CatalogStorageFormat, schema: StructType, provider: Option[String] = scala.None, partitionColumnNames: Seq[String] = collection.this.Seq.empty[Nothing], bucketSpec: Option[BucketSpec] = scala.None, owner: String = "", createTime: Long = ..., lastAccessTime: Long = -1, properties: Map[String, String] = ..., stats: Option[CatalogStatistics] = scala.None, viewText: Option[String] = scala.None, comment: Option[String] = scala.None, unsupportedFeatures: Seq[String] = collection.this.Seq.empty[Nothing], tracksPartitionsInCatalog: Boolean = false, schemaPreservesCase: Boolean = true) extends Product with Serializable

A table defined in the catalog.

Note that Hive's metastore also tracks skewed columns. We should consider adding that in the future once we have a better understanding of how we want to handle skewed columns.

provider

the name of the data source provider for this table, e.g. parquet, json, etc. Can be None if this table is a View, should be "hive" for hive serde tables.

unsupportedFeatures

is a list of string descriptions of features that are used by the underlying table but not supported by Spark SQL yet.

tracksPartitionsInCatalog

whether this table's partition metadata is stored in the catalog. If false, it is inferred automatically based on file structure.

schemaPreservesCase

Whether or not the schema resolved for this table is case-sensitive. When using a Hive Metastore, this flag is set to false if a case- sensitive schema was unable to be read from the table properties. Used to trigger case-sensitive schema inference at query time, when configured.

Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. CatalogTable
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CatalogTable(identifier: TableIdentifier, tableType: CatalogTableType, storage: CatalogStorageFormat, schema: StructType, provider: Option[String] = scala.None, partitionColumnNames: Seq[String] = collection.this.Seq.empty[Nothing], bucketSpec: Option[BucketSpec] = scala.None, owner: String = "", createTime: Long = ..., lastAccessTime: Long = -1, properties: Map[String, String] = ..., stats: Option[CatalogStatistics] = scala.None, viewText: Option[String] = scala.None, comment: Option[String] = scala.None, unsupportedFeatures: Seq[String] = collection.this.Seq.empty[Nothing], tracksPartitionsInCatalog: Boolean = false, schemaPreservesCase: Boolean = true)

    provider

    the name of the data source provider for this table, e.g. parquet, json, etc. Can be None if this table is a View, should be "hive" for hive serde tables.

    unsupportedFeatures

    is a list of string descriptions of features that are used by the underlying table but not supported by Spark SQL yet.

    tracksPartitionsInCatalog

    whether this table's partition metadata is stored in the catalog. If false, it is inferred automatically based on file structure.

    schemaPreservesCase

    Whether or not the schema resolved for this table is case-sensitive. When using a Hive Metastore, this flag is set to false if a case- sensitive schema was unable to be read from the table properties. Used to trigger case-sensitive schema inference at query time, when configured.

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. val bucketSpec: Option[BucketSpec]

  8. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  9. val comment: Option[String]

  10. val createTime: Long

  11. def dataSchema: StructType

    schema of this table's data columns

  12. def database: String

    Return the database this table was specified to belong to, assuming it exists.

  13. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  15. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  16. val identifier: TableIdentifier

  17. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  18. val lastAccessTime: Long

  19. def location: URI

    Return the table location, assuming it is specified.

  20. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  21. final def notify(): Unit

    Definition Classes
    AnyRef

  22. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  23. val owner: String

  24. val partitionColumnNames: Seq[String]

  25. def partitionSchema: StructType

    schema of this table's partition columns

  26. val properties: Map[String, String]

  27. val provider: Option[String]

    the name of the data source provider for this table, e.

    the name of the data source provider for this table, e.g. parquet, json, etc. Can be None if this table is a View, should be "hive" for hive serde tables.

  28. def qualifiedName: String

    Return the fully qualified name of this table, assuming the database was specified.

  29. val schema: StructType

  30. val schemaPreservesCase: Boolean

    Whether or not the schema resolved for this table is case-sensitive.

    Whether or not the schema resolved for this table is case-sensitive. When using a Hive Metastore, this flag is set to false if a case- sensitive schema was unable to be read from the table properties. Used to trigger case-sensitive schema inference at query time, when configured.

  31. def simpleString: String

    Readable string representation for the CatalogTable.

  32. val stats: Option[CatalogStatistics]

  33. val storage: CatalogStorageFormat

  34. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  35. val tableType: CatalogTableType

  36. def toLinkedHashMap: LinkedHashMap[String, String]

  37. def toString(): String

    Definition Classes
    CatalogTable → AnyRef → Any

  38. val tracksPartitionsInCatalog: Boolean

    whether this table's partition metadata is stored in the catalog.

    whether this table's partition metadata is stored in the catalog. If false, it is inferred automatically based on file structure.

  39. val unsupportedFeatures: Seq[String]

    is a list of string descriptions of features that are used by the underlying table but not supported by Spark SQL yet.

  40. def viewDefaultDatabase: Option[String]

    Return the default database name we use to resolve a view, should be None if the CatalogTable is not a View or created by older versions of Spark(before 2.

    Return the default database name we use to resolve a view, should be None if the CatalogTable is not a View or created by older versions of Spark(before 2.2.0).

  41. def viewQueryColumnNames: Seq[String]

    Return the output column names of the query that creates a view, the column names are used to resolve a view, should be empty if the CatalogTable is not a View or created by older versions of Spark(before 2.

    Return the output column names of the query that creates a view, the column names are used to resolve a view, should be empty if the CatalogTable is not a View or created by older versions of Spark(before 2.2.0).

  42. val viewText: Option[String]

  43. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  44. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  45. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  46. def withNewStorage(locationUri: Option[URI] = storage.locationUri, inputFormat: Option[String] = storage.inputFormat, outputFormat: Option[String] = storage.outputFormat, compressed: Boolean = false, serde: Option[String] = storage.serde, properties: Map[String, String] = storage.properties): CatalogTable

    Syntactic sugar to update a field in storage.

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped