org.apache.hadoop.hive.serde2.teradata

Class TeradataBinaryDataOutputStream

java.lang.Object

java.io.OutputStream

org.apache.commons.io.output.AbstractByteArrayOutputStream

org.apache.commons.io.output.ByteArrayOutputStream

org.apache.hadoop.hive.serde2.teradata.TeradataBinaryDataOutputStream

All Implemented Interfaces:

Closeable, Flushable, AutoCloseable

public class TeradataBinaryDataOutputStream
extends org.apache.commons.io.output.ByteArrayOutputStream

The TeradataBinaryDataOutputStream is used to produce the output in compliance with the Teradata binary format, so the output can be directly used to load into Teradata DB using TPT fastload. Since the TD binary format uses little-endian to handle the SHORT, INT, LONG, DOUBLE and etc. while the Hadoop uses big-endian, We extend SwappedDataInputStream to return qualified bytes for these types and extend to handle the Teradata specific types like VARCHAR, CHAR, TIMESTAMP, DATE...

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.commons.io.output.AbstractByteArrayOutputStream

org.apache.commons.io.output.AbstractByteArrayOutputStream.InputStreamConstructor<T extends InputStream>

Field Summary

Fields inherited from class org.apache.commons.io.output.AbstractByteArrayOutputStream

count

Constructor Summary

Constructors

Constructor and Description
TeradataBinaryDataOutputStream()

Method Summary

Modifier and Type	Method and Description
void	writeChar(HiveCharWritable writable, int length) Write CHAR(N).
void	writeDate(DateWritableV2 date) Write DATE.
void	writeDecimal(HiveDecimalWritable writable, int byteNum, int scale) Write DECIMAL(P, S).
void	writeDouble(double d) Write DOUBLE.
void	writeInt(int i) Write INT.
void	writeLong(long l) Write LONG.
void	writeShort(short s) Write SHORT.
void	writeTimestamp(TimestampWritableV2 timestamp, int byteNum) Write TIMESTAMP(N).
void	writeVarByte(org.apache.hadoop.io.BytesWritable writable) Write VARBYTE(N).
void	writeVarChar(HiveVarcharWritable writable) Write VARCHAR(N).

Methods inherited from class org.apache.commons.io.output.ByteArrayOutputStream

reset, size, toBufferedInputStream, toBufferedInputStream, toByteArray, toInputStream, write, write, write, writeTo

Methods inherited from class org.apache.commons.io.output.AbstractByteArrayOutputStream

close, needNewBuffer, resetImpl, toByteArrayImpl, toInputStream, toString, toString, toString, writeImpl, writeImpl, writeImpl, writeToImpl

Methods inherited from class java.io.OutputStream

flush, write

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

TeradataBinaryDataOutputStream

public TeradataBinaryDataOutputStream()

Method Detail

writeVarChar

public void writeVarChar(HiveVarcharWritable writable)
                  throws IOException

Write VARCHAR(N). The representation of Varchar in Teradata binary format is: the first two bytes represent the length N of this varchar field, the next N bytes represent the content of this varchar field. To pad the null varchar, the length will be 0 and the content will be none.

Parameters:

writable - the writable

Throws:

IOException - the io exception

writeInt

public void writeInt(int i)
              throws IOException

Write INT. using little-endian to write integer.

Parameters:

i - the

Throws:

IOException - the io exception

writeTimestamp

public void writeTimestamp(TimestampWritableV2 timestamp,
                           int byteNum)
                    throws IOException

Write TIMESTAMP(N). The representation of timestamp in Teradata binary format is: the byte number to read is based on the precision of timestamp, each byte represents one char and the timestamp is using string representation, eg: for 1911-11-11 19:20:21.433200 in TIMESTAMP(3), we will cut it to be 1911-11-11 19:20:21.433 and write 31 39 31 31 2d 31 31 2d 31 31 20 31 39 3a 32 30 3a 32 31 2e 34 33 33. the null timestamp will use space to pad.

Parameters:

timestamp - the timestamp

byteNum - the byte number the timestamp will write

Throws:

IOException - the io exception

writeDouble

public void writeDouble(double d)
                 throws IOException

Write DOUBLE. using little-endian to write double.

Parameters:

d - the d

Throws:

IOException - the io exception

writeDate

public void writeDate(DateWritableV2 date)
               throws IOException

Write DATE. The representation of date in Teradata binary format is: The Date D is a int with 4 bytes using little endian. The representation is (YYYYMMDD - 19000000).toInt -> D eg. 1911.11.11 -> 19111111 -> 111111 -> 07 b2 01 00 in little endian. the null date will use 0 to pad.

Parameters:

date - the date

Throws:

IOException - the io exception

writeLong

public void writeLong(long l)
               throws IOException

Write LONG. using little-endian to write double.

Parameters:

l - the l

Throws:

IOException - the io exception

writeChar

public void writeChar(HiveCharWritable writable,
                      int length)
               throws IOException

Write CHAR(N). The representation of char in Teradata binary format is: the byte number to read is based on the [charLength] * [bytePerChar] <- totalLength, bytePerChar is decided by the charset: LATIN charset is 2 bytes per char and UNICODE charset is 3 bytes per char. the null char will use space to pad.

Parameters:

writable - the writable

length - the byte n

Throws:

IOException - the io exception

writeDecimal

public void writeDecimal(HiveDecimalWritable writable,
                         int byteNum,
                         int scale)
                  throws IOException

Write DECIMAL(P, S). The representation of decimal in Teradata binary format is: the byte number to read is decided solely by the precision(P), HiveDecimal is constructed through the byte array and scale. the rest of byte will use 0x00 to pad (positive) and use 0xFF to pad (negative). the null DECIMAL will use 0x00 to pad.

Parameters:

writable - the writable

byteNum - the byte num

Throws:

IOException - the io exception

writeShort

public void writeShort(short s)
                throws IOException

Write SHORT. using little-endian to write short.

Parameters:

s - the s

Throws:

IOException - the io exception

writeVarByte

public void writeVarByte(org.apache.hadoop.io.BytesWritable writable)
                  throws IOException

Write VARBYTE(N). The representation of VARBYTE in Teradata binary format is: the first two bytes represent the length N of this varchar field, the next N bytes represent the content of this varchar field. To pad the null varbyte, the length will be 0 and the content will be none.

Parameters:

writable - the writable

Throws:

IOException - the io exception