org.apache.hadoop.hive.serde2.teradata

Class TeradataBinaryDataInputStream

java.lang.Object

java.io.InputStream

java.io.FilterInputStream

org.apache.commons.io.input.ProxyInputStream

org.apache.commons.io.input.SwappedDataInputStream

org.apache.hadoop.hive.serde2.teradata.TeradataBinaryDataInputStream

All Implemented Interfaces:

Closeable, DataInput, AutoCloseable

public class TeradataBinaryDataInputStream
extends org.apache.commons.io.input.SwappedDataInputStream

The TeradataBinaryDataInputStream is used to handle the Teradata binary format input for record. Since the TD binary format uses little-endian to handle the SHORT, INT, LONG, DOUBLE and etc. while the Hadoop uses big-endian, We extend SwappedDataInputStream to handle these types and extend to handle the Teradata specific types like VARCHAR, CHAR, TIMESTAMP, DATE...

Field Summary

Fields inherited from class java.io.FilterInputStream

in

Constructor Summary

Constructors

Constructor and Description
TeradataBinaryDataInputStream(InputStream input) Instantiates a new Teradata binary data input stream.

Method Summary

Modifier and Type	Method and Description
String	readChar(int totalLength) Read CHAR(N).
Date	readDate() Read DATE.
HiveDecimal	readDecimal(int scale, int byteNum) Read DECIMAL(P, S).
Timestamp	readTimestamp(Integer byteNum) Read TIMESTAMP(P).
byte[]	readVarbyte() Read VARBYTE(N).
String	readVarchar() Read VARCHAR(N).

Methods inherited from class org.apache.commons.io.input.SwappedDataInputStream

readBoolean, readByte, readChar, readDouble, readFloat, readFully, readFully, readInt, readLine, readLong, readShort, readUnsignedByte, readUnsignedShort, readUTF, skipBytes

Methods inherited from class org.apache.commons.io.input.ProxyInputStream

afterRead, available, beforeRead, close, handleIOException, mark, markSupported, read, read, read, reset, skip

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TeradataBinaryDataInputStream

public TeradataBinaryDataInputStream(InputStream input)

Instantiates a new Teradata binary data input stream.

Parameters:

input - the input

Method Detail

readVarchar

public String readVarchar()
                   throws IOException

Read VARCHAR(N). The representation of Varchar in Teradata binary format is: the first two bytes represent the length N of this varchar field, the next N bytes represent the content of this varchar field. To pad the null varchar, the length will be 0 and the content will be none.

Returns:

the string

Throws:

IOException - the io exception

readTimestamp

public Timestamp readTimestamp(Integer byteNum)
                        throws IOException

Read TIMESTAMP(P). The representation of timestamp in Teradata binary format is: the byte number to read is based on the precision of timestamp, each byte represents one char and the timestamp is using string representation, eg: for TIMESTAMP(6), we need to read 26 bytes 31 39 31 31 2d 31 31 2d 31 31 20 31 39 3a 32 30 3a 32 31 2e 34 33 33 32 30 30 will represent 1911-11-11 19:20:21.433200. the null timestamp will use space to pad.

Parameters:

byteNum - the byte number that will be read from inputstream

Returns:

the timestamp

Throws:

IOException - the io exception

readDate

public Date readDate()
              throws IOException,
                     ParseException

Read DATE. The representation of date in Teradata binary format is: The Date D is a int with 4 bytes using little endian, The representation is (D+19000000).ToString -> YYYYMMDD, eg: Date 07 b2 01 00 -> 111111 in little endian -> 19111111 - > 1911.11.11. the null date will use 0 to pad.

Returns:

the date

Throws:

IOException - the io exception

ParseException - the parse exception

readChar

public String readChar(int totalLength)
                throws IOException

Read CHAR(N). The representation of char in Teradata binary format is the byte number to read is based on the [charLength] * [bytePerChar] <- totalLength, bytePerChar is decided by the charset: LATIN charset is 2 bytes per char and UNICODE charset is 3 bytes per char. the null char will use space to pad.

Parameters:

totalLength - the total length

Returns:

the string

Throws:

IOException - the io exception

readDecimal

public HiveDecimal readDecimal(int scale,
                               int byteNum)
                        throws IOException

Read DECIMAL(P, S). The representation of decimal in Teradata binary format is the byte number to read is decided solely by the precision(P), HiveDecimal is constructed through the byte array and scale. the null DECIMAL will use 0x00 to pad.

Parameters:

scale - the scale

byteNum - the byte num

Returns:

the hive decimal

Throws:

IOException - the io exception

readVarbyte

public byte[] readVarbyte()
                   throws IOException

Read VARBYTE(N). The representation of VARBYTE in Teradata binary format is: the first two bytes represent the length N of this varchar field the next N bytes represent the content of this varchar field. To pad the null varbyte, the length will be 0 and the content will be none.

Returns:

the byte [ ]

Throws:

IOException - the io exception