org.apache.hadoop.hive.serde2.lazy

Class LazySimpleSerDe

java.lang.Object

org.apache.hadoop.hive.serde2.AbstractSerDe

org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe

org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe

All Implemented Interfaces:

Deserializer, SerDe, Serializer

Direct Known Subclasses:

DelimitedJSONSerDe

@InterfaceAudience.Public
 @InterfaceStability.Stable
public class LazySimpleSerDe
extends AbstractEncodingAwareSerDe

LazySimpleSerDe can be used to read the same data format as MetadataTypedColumnsetSerDe and TCTLSeparatedProtocol. However, LazySimpleSerDe creates Objects in a lazy way, to provide better performance. Also LazySimpleSerDe outputs typed columns instead of treating all columns as String like MetadataTypedColumnsetSerDe.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LazySimpleSerDe.SerDeParameters Deprecated.

Field Summary

Fields inherited from class org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe

charset

Fields inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

configuration, log, partitionProperties, properties, tableProperties

Constructor Summary

Constructors

Constructor and Description
LazySimpleSerDe()

Method Summary

Modifier and Type	Method and Description
Object	doDeserialize(org.apache.hadoop.io.Writable field) Deserialize a row from the Writable to a LazyObject.
org.apache.hadoop.io.Writable	doSerialize(Object obj, ObjectInspector objInspector) Serialize a row of data.
ObjectInspector	getObjectInspector() Returns the ObjectInspector for the row.
SerDeStats	getSerDeStats() Returns the statistics after (de)serialization)
Class<? extends org.apache.hadoop.io.Writable>	getSerializedClass() Returns the Writable Class after serialization.
void	initialize(org.apache.hadoop.conf.Configuration configuration, Properties tableProperties, Properties partitionProperties) Initialize the SerDe given the parameters.
static LazySimpleSerDe.SerDeParameters	initSerdeParams(org.apache.hadoop.conf.Configuration job, Properties tbl, String serdeName) Deprecated.
protected List<TypeInfo>	parseColumnTypes() NOTE: if "columns.types" is missing, all columns will be of String type.
static void	serialize(ByteStream.Output out, Object obj, ObjectInspector objInspector, byte[] separators, int level, org.apache.hadoop.io.Text nullSequence, boolean escaped, byte escapeChar, boolean[] needsEscape) Serialize the row into the StringBuilder.
protected void	serializeField(ByteStream.Output out, Object obj, ObjectInspector objInspector, LazySerDeParameters serdeParams)
String	toString()
protected org.apache.hadoop.io.Writable	transformFromUTF8(org.apache.hadoop.io.Writable blob) transform Writable data from UTF-8 to charset before serialize.
protected org.apache.hadoop.io.Writable	transformToUTF8(org.apache.hadoop.io.Writable blob) transform Writable data from charset to UTF-8 before doDeserialize.

Methods inherited from class org.apache.hadoop.hive.serde2.AbstractEncodingAwareSerDe

deserialize, serialize

Methods inherited from class org.apache.hadoop.hive.serde2.AbstractSerDe

getColumnComments, getColumnNames, getColumnTypes, getConfiguration, getPartitionColumnComments, getPartitionColumnNames, getPartitionColumnTypes, parseColumnComments, parseColumnNames, parseColumnNames, parseColumnTypes, shouldStoreFieldsInMetastore

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hive.serde2.SerDe

handleJobLevelConfiguration

Constructor Detail

LazySimpleSerDe

public LazySimpleSerDe()

Method Detail

toString

public String toString()

Overrides:

toString in class AbstractSerDe

initialize

public void initialize(org.apache.hadoop.conf.Configuration configuration,
                       Properties tableProperties,
                       Properties partitionProperties)
                throws SerDeException

Initialize the SerDe given the parameters. serialization.format: separator char or byte code (only supports byte-value up to 127) columns: ","-separated column names columns.types: ",", ":", or ";"-separated column types

Overrides:

initialize in class AbstractEncodingAwareSerDe

Parameters:

configuration - Hadoop configuration

tableProperties - Table properties

partitionProperties - Partition properties (may be null if table has no partitions)

Throws:

SerDeException - if SerDe fails to initialize

parseColumnTypes

protected List<TypeInfo> parseColumnTypes()

NOTE: if "columns.types" is missing, all columns will be of String type.

Overrides:

parseColumnTypes in class AbstractSerDe

doDeserialize

public Object doDeserialize(org.apache.hadoop.io.Writable field)
                     throws SerDeException

Deserialize a row from the Writable to a LazyObject.

Specified by:

doDeserialize in class AbstractEncodingAwareSerDe

Parameters:

field - the Writable that contains the data

Returns:

The deserialized row Object.

Throws:

SerDeException

See Also:

AbstractSerDe.deserialize(Writable)

getObjectInspector

public ObjectInspector getObjectInspector()
                                   throws SerDeException

Returns the ObjectInspector for the row.

Throws:

SerDeException

getSerializedClass

public Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()

Returns the Writable Class after serialization.

Specified by:

getSerializedClass in interface Serializer

Specified by:

getSerializedClass in class AbstractSerDe

See Also:

AbstractSerDe.getSerializedClass()

doSerialize

public org.apache.hadoop.io.Writable doSerialize(Object obj,
                                                 ObjectInspector objInspector)
                                          throws SerDeException

Serialize a row of data.

Specified by:

doSerialize in class AbstractEncodingAwareSerDe

Parameters:

obj - The row object

objInspector - The ObjectInspector for the row object

Returns:

The serialized Writable object

Throws:

SerDeException

See Also:

AbstractSerDe.serialize(Object, ObjectInspector)

serializeField

protected void serializeField(ByteStream.Output out,
                              Object obj,
                              ObjectInspector objInspector,
                              LazySerDeParameters serdeParams)
                       throws SerDeException

Throws:

SerDeException

serialize

public static void serialize(ByteStream.Output out,
                             Object obj,
                             ObjectInspector objInspector,
                             byte[] separators,
                             int level,
                             org.apache.hadoop.io.Text nullSequence,
                             boolean escaped,
                             byte escapeChar,
                             boolean[] needsEscape)
                      throws IOException,
                             SerDeException

Serialize the row into the StringBuilder.

Parameters:

out - The StringBuilder to store the serialized data.

obj - The object for the current field.

objInspector - The ObjectInspector for the current Object.

separators - The separators array.

level - The current level of separator.

nullSequence - The byte sequence representing the NULL value.

escaped - Whether we need to escape the data when writing out

escapeChar - Which char to use as the escape char, e.g. '\\'

needsEscape - Which byte needs to be escaped for 256 bytes.

Throws:

IOException

SerDeException

getSerDeStats

public SerDeStats getSerDeStats()

Returns the statistics after (de)serialization)

Specified by:

getSerDeStats in interface SerDe

Overrides:

getSerDeStats in class AbstractSerDe

Returns:

A SerDeStats object or null if stats are not supported by this SerDe. Returns statistics collected when serializing

transformFromUTF8

protected org.apache.hadoop.io.Writable transformFromUTF8(org.apache.hadoop.io.Writable blob)

Description copied from class: AbstractEncodingAwareSerDe

transform Writable data from UTF-8 to charset before serialize.

Specified by:

transformFromUTF8 in class AbstractEncodingAwareSerDe

Returns:

transformToUTF8

protected org.apache.hadoop.io.Writable transformToUTF8(org.apache.hadoop.io.Writable blob)

Description copied from class: AbstractEncodingAwareSerDe

transform Writable data from charset to UTF-8 before doDeserialize.

Specified by:

transformToUTF8 in class AbstractEncodingAwareSerDe

Returns:

initSerdeParams

@Deprecated
public static LazySimpleSerDe.SerDeParameters initSerdeParams(org.apache.hadoop.conf.Configuration job,
                                                                          Properties tbl,
                                                                          String serdeName)
                                                                   throws SerDeException

Deprecated.

This method is deprecated and is only used for backward compatibility. Replaced by @see org.apache.hadoop.hive.serde2.lazy.LazySerDeParameters#LazySerDeParameters(Configuration, Properties, String)

Throws:

SerDeException