Packages

package hive

Support for running Spark SQL queries using functionality from Apache Hive (does not require an existing Hive installation). Supported Hive features include:

  • Using HiveQL to express queries.
  • Reading metadata from the Hive Metastore using HiveSerDes.
  • Hive UDFs, UDAs, UDTs

Note that this is a private package. All classes in this package are considered an internal API to Spark and are subject to change between minor releases.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. hive
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Package Members

  1. package execution
  2. package orc

Type Members

  1. class DetermineTableStats extends Rule[LogicalPlan]
  2. class HiveSessionResourceLoader extends SessionResourceLoader
  3. class HiveSessionStateBuilder extends BaseSessionStateBuilder

    Builder that produces a Hive-aware SessionState.

  4. case class HiveUDAFBuffer(buf: AggregationBuffer, canDoMerge: Boolean) extends Product with Serializable
  5. case class RelationConversions(sessionCatalog: HiveSessionCatalog) extends Rule[LogicalPlan] with Product with Serializable

    Relation conversion from metastore relations to data source relations for better performance

    Relation conversion from metastore relations to data source relations for better performance

    - When writing to non-partitioned Hive-serde Parquet/Orc tables - When scanning Hive-serde Parquet/ORC tables

    This rule must be run before all other DDL post-hoc resolution rules, i.e. PreprocessTableCreation, PreprocessTableInsertion, DataSourceAnalysis and HiveAnalysis.

  6. class ResolveHiveSerdeTable extends Rule[LogicalPlan]

    Determine the database, serde/format and schema of the Hive serde table, according to the storage properties.

Deprecated Type Members

  1. class HiveContext extends SQLContext with Logging

    An instance of the Spark SQL execution engine that integrates with data stored in Hive.

    An instance of the Spark SQL execution engine that integrates with data stored in Hive. Configuration for Hive is read from hive-site.xml on the classpath.

    Annotations
    @deprecated
    Deprecated

    (Since version 2.0.0) Use SparkSession.builder.enableHiveSupport instead

Value Members

  1. object HiveAnalysis extends Rule[LogicalPlan]

    Replaces generic operations with specific variants that are designed to work with Hive.

    Replaces generic operations with specific variants that are designed to work with Hive.

    Note that, this rule must be run after PreprocessTableCreation and PreprocessTableInsertion.

  2. object HiveExternalCatalog

Inherited from AnyRef

Inherited from Any

Ungrouped