org.apache.hadoop.hive.serde2

Class AbstractSerDe

java.lang.Object

org.apache.hadoop.hive.serde2.AbstractSerDe

All Implemented Interfaces:

Deserializer, SerDe, Serializer

Direct Known Subclasses:

AbstractEncodingAwareSerDe, AvroSerDe, BinarySortableSerDe, ColumnarSerDeBase, JsonSerDe, LazyBinarySerDe, MetadataTypedColumnsetSerDe, NullStructSerDe, OpenCSVSerde, RegexSerDe, TeradataBinarySerde, ThriftDeserializer, ThriftJDBCBinarySerDe, TypedSerDe

public abstract class AbstractSerDe
extends Object
implements Deserializer, Serializer

This abstract class is the superclass of all classes that can serialize and de-serialize Hadoop Writable objects.

Field Summary

Fields

Modifier and Type	Field and Description
protected Optional<org.apache.hadoop.conf.Configuration>	configuration
protected org.slf4j.Logger	log
protected Optional<Properties>	partitionProperties
protected Properties	properties
protected Properties	tableProperties

Constructor Summary

Constructors

Constructor and Description
AbstractSerDe()

Method Summary

Modifier and Type	Method and Description
Object	deserialize(org.apache.hadoop.io.Writable blob) Deserialize an object out of a Writable blob.
List<String>	getColumnComments()
List<String>	getColumnNames()
List<TypeInfo>	getColumnTypes()
Optional<org.apache.hadoop.conf.Configuration>	getConfiguration()
List<String>	getPartitionColumnComments()
List<String>	getPartitionColumnNames()
List<TypeInfo>	getPartitionColumnTypes()
SerDeStats	getSerDeStats() Returns statistics collected when serializing.
abstract Class<? extends org.apache.hadoop.io.Writable>	getSerializedClass() Returns the Writable class that would be returned by the serialize method.
void	initialize(org.apache.hadoop.conf.Configuration configuration, Properties tableProperties, Properties partitionProperties) Initialize the SerDe.
protected List<String>	parseColumnComments(String key)
protected List<String>	parseColumnNames()
protected List<String>	parseColumnNames(String key)
protected List<TypeInfo>	parseColumnTypes()
protected List<TypeInfo>	parseColumnTypes(String key)
org.apache.hadoop.io.Writable	serialize(Object obj, ObjectInspector objInspector) Serialize an object by navigating inside the Object with the ObjectInspector.
boolean	shouldStoreFieldsInMetastore(Map<String,String> tableParams)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hive.serde2.Deserializer

getObjectInspector

Methods inherited from interface org.apache.hadoop.hive.serde2.SerDe

handleJobLevelConfiguration

Field Detail

log

protected org.slf4j.Logger log

configuration

protected Optional<org.apache.hadoop.conf.Configuration> configuration

properties

protected Properties properties

tableProperties

protected Properties tableProperties

partitionProperties

protected Optional<Properties> partitionProperties

Constructor Detail

AbstractSerDe

public AbstractSerDe()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration configuration,
                       Properties tableProperties,
                       Properties partitionProperties)
                throws SerDeException

Initialize the SerDe. By default, this will use one set of properties, either the table properties or the partition properties. If a SerDe needs access to both sets, it should override this method. Eventually, once all SerDes have implemented this method, we should convert it to an abstract method.

Parameters:

configuration - Hadoop configuration

tableProperties - Table properties

partitionProperties - Partition properties (may be null if table has no partitions)

Throws:

NullPointerException - if tableProperties is null

SerDeException - if SerDe fails to initialize

parseColumnNames

protected List<String> parseColumnNames()

parseColumnNames

protected List<String> parseColumnNames(String key)

parseColumnTypes

protected List<TypeInfo> parseColumnTypes()

parseColumnTypes

protected List<TypeInfo> parseColumnTypes(String key)

parseColumnComments

protected List<String> parseColumnComments(String key)

getSerializedClass

public abstract Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()

Returns the Writable class that would be returned by the serialize method. This is used to initialize SequenceFile header.

Specified by:

getSerializedClass in interface Serializer

getSerDeStats

public SerDeStats getSerDeStats()

Returns statistics collected when serializing.

Specified by:

getSerDeStats in interface SerDe

Returns:

A SerDeStats object or null if stats are not supported by this SerDe. Returns statistics collected when serializing

serialize

public org.apache.hadoop.io.Writable serialize(Object obj,
                                               ObjectInspector objInspector)
                                        throws SerDeException

Description copied from interface: Serializer

Serialize an object by navigating inside the Object with the ObjectInspector. In most cases, the return value of this function will be constant since the function will reuse the Writable object. If the client wants to keep a copy of the Writable, the client needs to clone the returned value.

Specified by:

serialize in interface Serializer

Throws:

SerDeException

deserialize

public Object deserialize(org.apache.hadoop.io.Writable blob)
                   throws SerDeException

Description copied from interface: Deserializer

Deserialize an object out of a Writable blob. In most cases, the return value of this function will be constant since the function will reuse the returned object. If the client wants to keep a copy of the object, the client needs to clone the returned deserialized value by calling ObjectInspectorUtils.getStandardObject().

Specified by:

deserialize in interface Deserializer

Parameters:

blob - The Writable object containing a serialized object

Returns:

A Java object representing the contents in the blob.

Throws:

SerDeException

shouldStoreFieldsInMetastore

public boolean shouldStoreFieldsInMetastore(Map<String,String> tableParams)

Returns:

Whether the SerDe that can store schema both inside and outside of metastore does, in fact, store it inside metastore, based on table parameters.

getColumnNames

public List<String> getColumnNames()

getColumnTypes

public List<TypeInfo> getColumnTypes()

getColumnComments

public List<String> getColumnComments()

getPartitionColumnNames

public List<String> getPartitionColumnNames()

getPartitionColumnTypes

public List<TypeInfo> getPartitionColumnTypes()

getPartitionColumnComments

public List<String> getPartitionColumnComments()

getConfiguration

public Optional<org.apache.hadoop.conf.Configuration> getConfiguration()

toString

public String toString()

Overrides:

toString in class Object