org.apache.hadoop.hive.serde2.avro

Class AvroSerDe

java.lang.Object

org.apache.hadoop.hive.serde2.AbstractSerDe

org.apache.hadoop.hive.serde2.avro.AvroSerDe

All Implemented Interfaces:

Deserializer, SerDe, Serializer

public class AvroSerDe
extends AbstractSerDe

Read or write Avro data from Hive.

Field Summary

Fields

Modifier and Type	Field and Description
static String	AVRO_INT_TYPE_NAME
static String	AVRO_LONG_TYPE_NAME
static String	AVRO_PROP_LOGICAL_TYPE
static String	AVRO_PROP_MAX_LENGTH
static String	AVRO_PROP_PRECISION
static String	AVRO_PROP_SCALE
static String	AVRO_STRING_TYPE_NAME
static String	CHAR_TYPE_NAME
static String	DATE_TYPE_NAME
static String	DECIMAL_TYPE_NAME
static String	LIST_COLUMN_COMMENTS
static String	TABLE_COMMENT
static String	TABLE_NAME
static String	TIMESTAMP_TYPE_NAME
static String	VARCHAR_TYPE_NAME
static String	WRITER_PROLEPTIC
static String	WRITER_TIME_ZONE
static String	WRITER_ZONE_CONVERSION_LEGACY

Fields inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

configuration, log, partitionProperties, properties, tableProperties

Constructor Summary

Constructors

Constructor and Description
AvroSerDe()

Method Summary

Modifier and Type	Method and Description
Object	deserialize(org.apache.hadoop.io.Writable writable) Deserialize an object out of a Writable blob.
ObjectInspector	getObjectInspector() Get the object inspector that can be used to navigate through the internal structure of the Object returned from deserialize(...).
static org.apache.avro.Schema	getSchemaFromCols(Properties properties, List<String> columnNames, List<TypeInfo> columnTypes, String columnCommentProperty)
Class<? extends org.apache.hadoop.io.Writable>	getSerializedClass() Returns the Writable class that would be returned by the serialize method.
void	initialize(org.apache.hadoop.conf.Configuration configuration, Properties tableProperties, Properties partitionProperties) Initialize the SerDe.
org.apache.hadoop.io.Writable	serialize(Object o, ObjectInspector objectInspector) Serialize an object by navigating inside the Object with the ObjectInspector.
boolean	shouldStoreFieldsInMetastore(Map<String,String> tableParams)

Methods inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

getColumnComments, getColumnNames, getColumnTypes, getConfiguration, getPartitionColumnComments, getPartitionColumnNames, getPartitionColumnTypes, getSerDeStats, parseColumnComments, parseColumnNames, parseColumnNames, parseColumnTypes, parseColumnTypes, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hive.serde2.SerDe

handleJobLevelConfiguration

Field Detail

TABLE_NAME

public static final String TABLE_NAME

See Also:

Constant Field Values

TABLE_COMMENT

public static final String TABLE_COMMENT

See Also:

Constant Field Values

LIST_COLUMN_COMMENTS

public static final String LIST_COLUMN_COMMENTS

See Also:

Constant Field Values

DECIMAL_TYPE_NAME

public static final String DECIMAL_TYPE_NAME

See Also:

Constant Field Values

CHAR_TYPE_NAME

public static final String CHAR_TYPE_NAME

See Also:

Constant Field Values

VARCHAR_TYPE_NAME

public static final String VARCHAR_TYPE_NAME

See Also:

Constant Field Values

DATE_TYPE_NAME

public static final String DATE_TYPE_NAME

See Also:

Constant Field Values

TIMESTAMP_TYPE_NAME

public static final String TIMESTAMP_TYPE_NAME

See Also:

Constant Field Values

WRITER_TIME_ZONE

public static final String WRITER_TIME_ZONE

See Also:

Constant Field Values

WRITER_PROLEPTIC

public static final String WRITER_PROLEPTIC

See Also:

Constant Field Values

WRITER_ZONE_CONVERSION_LEGACY

public static final String WRITER_ZONE_CONVERSION_LEGACY

See Also:

Constant Field Values

AVRO_PROP_LOGICAL_TYPE

public static final String AVRO_PROP_LOGICAL_TYPE

See Also:

Constant Field Values

AVRO_PROP_PRECISION

public static final String AVRO_PROP_PRECISION

See Also:

Constant Field Values

AVRO_PROP_SCALE

public static final String AVRO_PROP_SCALE

See Also:

Constant Field Values

AVRO_PROP_MAX_LENGTH

public static final String AVRO_PROP_MAX_LENGTH

See Also:

Constant Field Values

AVRO_STRING_TYPE_NAME

public static final String AVRO_STRING_TYPE_NAME

See Also:

Constant Field Values

AVRO_INT_TYPE_NAME

public static final String AVRO_INT_TYPE_NAME

See Also:

Constant Field Values

AVRO_LONG_TYPE_NAME

public static final String AVRO_LONG_TYPE_NAME

See Also:

Constant Field Values

Constructor Detail

AvroSerDe

public AvroSerDe()

Method Detail

initialize

public void initialize(org.apache.hadoop.conf.Configuration configuration,
                       Properties tableProperties,
                       Properties partitionProperties)
                throws SerDeException

Description copied from class: AbstractSerDe

Initialize the SerDe. By default, this will use one set of properties, either the table properties or the partition properties. If a SerDe needs access to both sets, it should override this method. Eventually, once all SerDes have implemented this method, we should convert it to an abstract method.

Overrides:

initialize in class AbstractSerDe

Parameters:

configuration - Hadoop configuration

tableProperties - Table properties

partitionProperties - Partition properties (may be null if table has no partitions)

Throws:

SerDeException - if SerDe fails to initialize

getSchemaFromCols

public static org.apache.avro.Schema getSchemaFromCols(Properties properties,
                                                       List<String> columnNames,
                                                       List<TypeInfo> columnTypes,
                                                       String columnCommentProperty)

getSerializedClass

public Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()

Description copied from class: AbstractSerDe

Returns the Writable class that would be returned by the serialize method. This is used to initialize SequenceFile header.

Specified by:

getSerializedClass in interface Serializer

Specified by:

getSerializedClass in class AbstractSerDe

serialize

public org.apache.hadoop.io.Writable serialize(Object o,
                                               ObjectInspector objectInspector)
                                        throws SerDeException

Description copied from interface: Serializer

Serialize an object by navigating inside the Object with the ObjectInspector. In most cases, the return value of this function will be constant since the function will reuse the Writable object. If the client wants to keep a copy of the Writable, the client needs to clone the returned value.

Specified by:

serialize in interface Serializer

Overrides:

serialize in class AbstractSerDe

Throws:

SerDeException

deserialize

public Object deserialize(org.apache.hadoop.io.Writable writable)
                   throws SerDeException

Description copied from interface: Deserializer

Deserialize an object out of a Writable blob. In most cases, the return value of this function will be constant since the function will reuse the returned object. If the client wants to keep a copy of the object, the client needs to clone the returned deserialized value by calling ObjectInspectorUtils.getStandardObject().

Specified by:

deserialize in interface Deserializer

Overrides:

deserialize in class AbstractSerDe

Parameters:

writable - The Writable object containing a serialized object

Returns:

A Java object representing the contents in the blob.

Throws:

SerDeException

getObjectInspector

public ObjectInspector getObjectInspector()
                                   throws SerDeException

Description copied from interface: Deserializer

Get the object inspector that can be used to navigate through the internal structure of the Object returned from deserialize(...).

Throws:

SerDeException

shouldStoreFieldsInMetastore

public boolean shouldStoreFieldsInMetastore(Map<String,String> tableParams)

Overrides:

shouldStoreFieldsInMetastore in class AbstractSerDe

Returns:

Whether the SerDe that can store schema both inside and outside of metastore does, in fact, store it inside metastore, based on table parameters.