org.apache.spark.sql

types

package types

Contains a type system for attributes produced by relations, including complex types like structs, arrays and maps.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. types
  2. AnyRef
  3. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. case class ArrayType(elementType: DataType, containsNull: Boolean) extends DataType with Product with Serializable

    The data type for collections of multiple values.

  2. abstract class AtomicType extends DataType

    An internal type used to represent everything that is not null, UDTs, arrays, structs, and maps.

  3. class BinaryType extends AtomicType

    The data type representing Array[Byte] values.

  4. class BooleanType extends AtomicType

    The data type representing Boolean values.

  5. class ByteType extends IntegralType

    The data type representing Byte values.

  6. class CalendarIntervalType extends DataType

    The data type representing calendar time intervals.

  7. case class CharType(length: Int) extends HiveStringType with Product with Serializable

    Hive char type.

  8. abstract class DataType extends AbstractDataType

    The base type of all Spark SQL data types.

  9. class DataTypes extends AnyRef

  10. class DateType extends AtomicType

    A date type, supporting "0001-01-01" through "9999-12-31".

  11. final class Decimal extends Ordered[Decimal] with Serializable

    A mutable implementation of BigDecimal that can hold a Long if values are small enough.

  12. case class DecimalType(precision: Int, scale: Int) extends FractionalType with Product with Serializable

    The data type representing java.math.BigDecimal values.

  13. class DoubleType extends FractionalType

    The data type representing Double values.

  14. class FloatType extends FractionalType

    The data type representing Float values.

  15. sealed abstract class HiveStringType extends AtomicType

    A hive string type for compatibility.

  16. class IntegerType extends IntegralType

    The data type representing Int values.

  17. class LongType extends IntegralType

    The data type representing Long values.

  18. case class MapType(keyType: DataType, valueType: DataType, valueContainsNull: Boolean) extends DataType with Product with Serializable

    The data type for Maps.

  19. sealed class Metadata extends Serializable

    Metadata is a wrapper over Map[String, Any] that limits the value type to simple ones: Boolean, Long, Double, String, Metadata, Array[Boolean], Array[Long], Array[Double], Array[String], and Array[Metadata].

  20. class MetadataBuilder extends AnyRef

    Builder for Metadata.

  21. class NullType extends DataType

    The data type representing NULL values.

  22. abstract class NumericType extends AtomicType

    Numeric data types.

  23. case class ObjectType(cls: Class[_]) extends DataType with Product with Serializable

    Represents a JVM object that is passing through Spark SQL expression evaluation.

  24. class SQLUserDefinedType extends Annotation with Annotation with ClassfileAnnotation

  25. class ShortType extends IntegralType

    The data type representing Short values.

  26. class StringType extends AtomicType

    The data type representing String values.

  27. case class StructField(name: String, dataType: DataType, nullable: Boolean = true, metadata: Metadata = Metadata.empty) extends Product with Serializable

    A field inside a StructType.

  28. case class StructType(fields: Array[StructField]) extends DataType with Seq[StructField] with Product with Serializable

    A StructType object can be constructed by

  29. class TimestampType extends AtomicType

    The data type representing java.sql.Timestamp values.

  30. case class VarcharType(length: Int) extends HiveStringType with Product with Serializable

    Hive varchar type.

Value Members

  1. object AnyDataType extends AbstractDataType

    An AbstractDataType that matches any concrete data types.

  2. object ArrayType extends AbstractDataType with Serializable

    Companion object for ArrayType.

  3. object AtomicType

  4. object BinaryType extends BinaryType with Product with Serializable

  5. object BooleanType extends BooleanType with Product with Serializable

  6. object ByteType extends ByteType with Product with Serializable

  7. object CalendarIntervalType extends CalendarIntervalType with Product with Serializable

  8. object DataType

  9. object DateType extends DateType with Product with Serializable

  10. object Decimal extends Serializable

    Annotations
    @Unstable()

  11. object DecimalType extends AbstractDataType with Serializable

    Extra factory methods and pattern matchers for Decimals.

  12. object DoubleType extends DoubleType with Product with Serializable

  13. object FloatType extends FloatType with Product with Serializable

  14. val HIVE_TYPE_STRING: String

    Metadata key used to store the raw hive type string in the metadata of StructField.

    Metadata key used to store the raw hive type string in the metadata of StructField. This is relevant for datatypes that do not have a direct Spark SQL counterpart, such as CHAR and VARCHAR. We need to preserve the original type in order to invoke the correct object inspector in Hive.

  15. object HiveStringType

  16. object IntegerType extends IntegerType with Product with Serializable

  17. object LongType extends LongType with Product with Serializable

  18. object MapType extends AbstractDataType with Serializable

  19. object Metadata extends Serializable

  20. object NullType extends NullType with Product with Serializable

  21. object ObjectType extends AbstractDataType with Serializable

    Annotations
    @Evolving()

  22. object ShortType extends ShortType with Product with Serializable

  23. object StringType extends StringType with Product with Serializable

  24. object StructType extends AbstractDataType with Serializable

  25. object TimestampType extends TimestampType with Product with Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped