object FileSourceMetadataAttribute
The internal representation of the FileSourceMetadataAttribute, it sets metadata_col to
and file_source_metadata_coltrue in AttributeReference's metadata.
This is a super type of FileSourceConstantMetadataAttribute and
FileSourceGeneratedMetadataAttribute.
- apply() will create a file source metadata attribute reference - unapply() will check if an attribute reference is any file source metadata attribute reference
- Alphabetic
- By Inheritance
- FileSourceMetadataAttribute
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- val FILE_SOURCE_METADATA_COL_ATTR_KEY: String
- def apply(name: String, dataType: DataType, nullable: Boolean = false): AttributeReference
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def cleanupFileSourceMetadataInformation(field: StructField): StructField
Removes the internal field metadata.
- def cleanupFileSourceMetadataInformation(attr: Attribute): Attribute
Removes the internal field metadata.
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def isSupportedType(dataType: DataType): Boolean
True if the given data type is supported in file source metadata attributes.
True if the given data type is supported in file source metadata attributes.
The set of supported types is limited by ColumnVectorUtils.populate, which the constant file metadata implementation relies on. In general, types that can be partition columns are supported (including most primitive types). Notably unsupported types include ObjectType, UserDefinedType, and the complex types (StructType, MapType, ArrayType).
- def isValid(metadata: Metadata): Boolean
- def metadata(name: String): Metadata
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def unapply(attr: AttributeReference): Option[AttributeReference]
Matches if attr is any File source metadata attribute (including constant and generated).
- def validateType(dataType: DataType): DataType
Returns the type unchanged if valid; otherwise throws IllegalArgumentException.
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()