org.apache.hadoop.hive.common.type

Class HiveDecimal

java.lang.Object

org.apache.hadoop.hive.common.type.FastHiveDecimal

org.apache.hadoop.hive.common.type.HiveDecimal

All Implemented Interfaces:

Comparable<HiveDecimal>

public final class HiveDecimal
extends FastHiveDecimal
implements Comparable<HiveDecimal>

HiveDecimal is a decimal data type with a maximum precision and scale.

It is the Hive DECIMAL data type.

The scale is the number of fractional decimal digits. The digits after the dot. It is limited to 38 (MAX_SCALE).

The precision is the integer (or whole-number) decimal digits plus fractional decimal digits. It is limited to a total of 38 digits (MAX_PRECISION).

Hive syntax for declaring DECIMAL has 3 forms:

DECIMAL // Use the default precision/scale.

DECIMAL(precision) // Use the default scale.

DECIMAL(precision, scale) }

The declared scale must be <= precision.

Use DECIMAL instead of DOUBLE when exact numeric accuracy is required. Not all decimal numbers (radix 10) are exactly representable in the binary (radix 2 based) floating point type DOUBLE and cause accuracy anomalies (i.e. wrong results). See the Internet for more details.

HiveDecimal is implemented as a classic Java immutable object. All operations on HiveDecimal that produce a different value will create a new HiveDecimal object.

Decimals are physically stored without any extra leading or trailing zeroes. The scale of a decimal is the number of non-trailing zero fractional digits.

Math operations on decimals typically cause the scale to change as a result of the math and from trailing fractional digit elimination.

Typically, Hive, when it wants to make sure a result decimal fits in the column decimal's precision/scale it calls enforcePrecisionScale. That method will scale down or trim off result fractional digits if necessary with rounding when the column has a smaller scale. And, it will also indicate overflow when the decimal has exceeded the column's maximum precision.

NOTE: When Hive gets ready to serialize a decimal into text or binary, it usually sometimes wants trailing fractional zeroes. See the special notes for toFormatString and bigIntegerBytesScaled for details.

------------------------------------- Version 2 ------------------------------------------------

This is the 2nd major version of HiveDecimal called V2. The previous version has been renamed to HiveDecimalV1 and is kept as a test and behavior reference.

For good performance we do not represent the decimal using a BigDecimal object like the previous version V1 did. Using Java objects to represent our decimal incurs too high a penalty for memory allocations and general logic.

The original V1 public methods and fields are annotated with @HiveDecimalVersionV1; new public methods and fields are annotated with @HiveDecimalVersionV2.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.common.type.FastHiveDecimal

FastHiveDecimal.FastCheckPrecisionScaleStatus

Field Summary

Fields

Modifier and Type	Field and Description
static int	MAX_PRECISION
static int	MAX_SCALE
static HiveDecimal	ONE
static int	ROUND_CEILING ROUND_CEILING:
static int	ROUND_FLOOR ROUND_FLOOR:
static int	ROUND_HALF_EVEN ROUND_HALF_EVEN: Round towards the "nearest neighbor" unless both neighbors are equidistant, then round towards the even neighbor.
static int	ROUND_HALF_UP ROUND_HALF_UP:
static int	SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES
static int	SCRATCH_BUFFER_LEN_SERIALIZATION_UTILS_READ
static int	SCRATCH_BUFFER_LEN_TO_BYTES
static int	SCRATCH_LONGS_LEN
static int	SYSTEM_DEFAULT_PRECISION Default precision/scale when system is not able to determine them, such as in case of a non-generic udf.
static int	SYSTEM_DEFAULT_SCALE
static int	USER_DEFAULT_PRECISION Default precision/scale when user doesn't specify in the column metadata, such as decimal and decimal(8).
static int	USER_DEFAULT_SCALE
static HiveDecimal	ZERO Common values.

Fields inherited from class org.apache.hadoop.hive.common.type.FastHiveDecimal

FAST_SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES, FAST_SCRATCH_BUFFER_LEN_SERIALIZATION_UTILS_READ, FAST_SCRATCH_BUFFER_LEN_TO_BYTES, FAST_SCRATCH_LONGS_LEN, fast0, fast1, fast2, fastIntegerDigitCount, fastScale, fastSerializationScale, fastSignum, SCRATCH_LONGS_LEN_FAST_SERIALIZATION_UTILS_WRITE, STRING_ENFORCE_PRECISION_OUT_OF_RANGE, STRING_ENFORCE_SCALE_LESS_THAN_EQUAL_PRECISION, STRING_ENFORCE_SCALE_OUT_OF_RANGE

Method Summary

Modifier and Type	Method and Description
HiveDecimal	abs() Take the absolute value of a decimal.
HiveDecimal	add(HiveDecimal dec) Add the current decimal and another decimal and return the result.
BigDecimal	bigDecimalValue() Return a BigDecimal representing the decimal.
byte[]	bigIntegerBytes()
int	bigIntegerBytes(long[] scratchLongs, byte[] buffer) Return binary representation of this decimal's BigInteger equivalent unscaled value using the format that the BigInteger's toByteArray method returns (and the BigInteger constructor accepts).
byte[]	bigIntegerBytesScaled(int serializeScale)
int	bigIntegerBytesScaled(int serializeScale, long[] scratchLongs, byte[] buffer) Convert decimal to BigInteger binary bytes with a serialize scale, similar to the formatScale for toFormatString.
byte	byteValue() A byte variation of longValue()
int	compareTo(HiveDecimal dec)
static HiveDecimal	create(BigDecimal bigDecimal) Create a HiveDecimal from BigDecimal object.
static HiveDecimal	create(BigDecimal bigDecimal, boolean allowRounding) Same as the above create method, except fractional digit rounding can be turned off.
static HiveDecimal	create(BigInteger bigInteger) Creates a HiveDecimal from a BigInteger's value with a scale of 0.
static HiveDecimal	create(BigInteger bigInteger, int scale) Creates a HiveDecimal from a BigInteger's value with a specified scale.
static HiveDecimal	create(boolean isNegative, byte[] bytes, int scale) This method takes in digits only UTF-8 characters, a sign flag, and a scale and returns a decimal.
static HiveDecimal	create(boolean isNegative, byte[] bytes, int offset, int length, int scale)
static HiveDecimal	create(byte[] bytes) Create a HiveDecimal by parsing the characters in a whole byte array.
static HiveDecimal	create(byte[] bytes, boolean trimBlanks) Same as the method above, except blanks before and after are tolerated.
static HiveDecimal	create(byte[] bytes, int offset, int length) Create a HiveDecimal by parsing the characters in a slice of a byte array.
static HiveDecimal	create(byte[] bytes, int offset, int length, boolean trimBlanks) Same as the method above, except blanks before and after are tolerated.
static HiveDecimal	create(double doubleValue) Create a HiveDecimal object from a double.
static HiveDecimal	create(float floatValue) Create a HiveDecimal object from a float.
static HiveDecimal	create(int intValue) Create a HiveDecimal object from an int.
static HiveDecimal	create(long longValue) Create a HiveDecimal object from a long.
static HiveDecimal	create(long longValue, int scale) Create a HiveDecimal object from a long with a specified scale.
static HiveDecimal	create(String string) Create a HiveDecimal by parsing a whole string.
static HiveDecimal	create(String string, boolean trimBlanks) Same as the method above, except blanks before and after are tolerated.
static HiveDecimal	createFromBigIntegerBytesAndScale(byte[] bytes, int scale) Convert bytes in the format used by BigInteger's toByteArray format (and accepted by its constructor) into a decimal using the specified scale.
static HiveDecimal	createFromBigIntegerBytesAndScale(byte[] bytes, int offset, int length, int scale)
static HiveDecimal	createFromFast(FastHiveDecimal fastDec) Create a HiveDecimal from a FastHiveDecimal object.
HiveDecimal	divide(HiveDecimal divisor) Divides this decimal by another decimal and returns a new decimal with the result.
double	doubleValue() Return a double representing the decimal.
static HiveDecimal	enforcePrecisionScale(HiveDecimal dec, int maxPrecision, int maxScale) Determine if a decimal fits within a specified maxPrecision and maxScale, and round off fractional digits if necessary to make the decimal fit.
boolean	equals(Object obj) Are two decimal content (values) equal?
float	floatValue() Return a float representing the decimal.
HiveDecimal	fractionPortion() Return a decimal with only the fractional digits.
int	hashCode() This is returns original hash code as returned by HiveDecimalV1.
int	integerDigitCount() Returns the number of integer digits in the decimal.
HiveDecimal	integerPortion() Return a decimal with only the integer digits.
int	intValue() An int variation of longValue().
boolean	isByte() Is the decimal value a byte? Range -128 to 127.
boolean	isInt() Is the decimal value a int? Range -2,147,483,648 to 2,147,483,647.
boolean	isLong() Is the decimal value a long? Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807.
boolean	isShort() Is the decimal value a short? Range -32,768 to 32,767.
long	longValue() Return the long value of a decimal.
long	longValueExact()
HiveDecimal	multiply(HiveDecimal dec) Multiply two decimals.
HiveDecimal	negate() Reverse the sign of a decimal.
int	newFasterHashCode() Hash code based on (new) decimal representation.
HiveDecimal	pow(int exponent) Return the result of decimal^exponent
int	precision() Returns the number of digits (integer and fractional) in the number, which is equivalent to SQL decimal precision.
int	rawPrecision()
HiveDecimal	remainder(HiveDecimal divisor) Divides this decimal by another decimal and returns a new decimal with the remainder of the division.
int	scale() Returns the scale of the decimal.
HiveDecimal	scaleByPowerOfTen(int power) Multiplies a decimal by a power of 10.
static HiveDecimal	serializationUtilsRead(InputStream inputStream, int scale, byte[] scratchBytes) Deserialize data written in the format used by the SerializationUtils methods readBigInteger/writeBigInteger and create a decimal using the supplied scale.
boolean	serializationUtilsWrite(OutputStream outputStream, long[] scratchLongs) Serialize this decimal's BigInteger equivalent unscaled value using the format that the SerializationUtils methods readBigInteger/writeBigInteger use.
HiveDecimal	setScale(int serializationScale) Deprecated.
HiveDecimal	setScale(int roundingPoint, int roundingMode) Do decimal rounding and return the result.
short	shortValue() A short variation of longValue().
int	signum() Get the sign of the decimal.
HiveDecimal	subtract(HiveDecimal dec) Subtract from the current decimal another decimal and return the result.
int	toBytes(byte[] scratchBuffer) Decimal to ASCII bytes conversion.
int	toDigitsOnlyBytes(byte[] scratchBuffer) Convert decimal to just the digits -- no dot.
String	toDigitsOnlyString()
int	toFormatBytes(int formatScale, byte[] scratchBuffer) This is the serialization version of decimal to string conversion.
String	toFormatString(int formatScale) Return a string representation of the decimal using the specified scale.
String	toFormatString(int formatScale, byte[] scratchBuffer)
String	toString() Return a string representation of the decimal.
String	toString(byte[] scratchBuffer)
BigInteger	unscaledValue() Get a BigInteger representing the decimal's digits without a dot.
void	validate() Throws an exception if the current decimal value is invalid.

Methods inherited from class org.apache.hadoop.hive.common.type.FastHiveDecimal

fastAbs, fastAdd, fastBigDecimalValue, fastBigIntegerBytes, fastBigIntegerBytesScaled, fastBigIntegerValue, fastByteValueClip, fastCheckPrecisionScale, fastCompareTo, fastCompareTo, fastDeserialize64, fastDivide, fastDoubleValue, fastEnforcePrecisionScale, fastEquals, fastFloatValue, fastFractionPortion, fastHashCode, fastIntegerDigitCount, fastIntegerPortion, fastIntValueClip, fastIsByte, fastIsInt, fastIsLong, fastIsShort, fastIsValid, fastLongValueClip, fastMultiply, fastNegate, fastNewFasterHashCode, fastPow, fastRaiseInvalidException, fastRaiseInvalidException, fastRawPrecision, fastRemainder, fastReset, fastRound, fastRoundingModeToString, fastScale, fastScaleByPowerOfTen, fastSerializationScale, fastSerializationUtilsRead, fastSerializationUtilsWrite, fastSerialize64, fastSet, fastSet, fastSetFromBigDecimal, fastSetFromBigInteger, fastSetFromBigIntegerAndScale, fastSetFromBigIntegerBytesAndScale, fastSetFromBytes, fastSetFromDigitsOnlyBytesAndScale, fastSetFromDouble, fastSetFromFloat, fastSetFromInt, fastSetFromLong, fastSetFromLongAndScale, fastSetFromString, fastSetSerializationScale, fastShortValueClip, fastSignum, fastSqlPrecision, fastSubtract, fastToBytes, fastToDigitsOnlyBytes, fastToDigitsOnlyString, fastToFormatBytes, fastToFormatString, fastToFormatString, fastToString, fastToString, fastUpdatePrecisionScale, isAllZeroesBelow

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

MAX_PRECISION

@HiveDecimalVersionV1
public static final int MAX_PRECISION

See Also:

Constant Field Values

MAX_SCALE

@HiveDecimalVersionV1
public static final int MAX_SCALE

See Also:

Constant Field Values

USER_DEFAULT_PRECISION

@HiveDecimalVersionV1
public static final int USER_DEFAULT_PRECISION

Default precision/scale when user doesn't specify in the column metadata, such as decimal and decimal(8).

See Also:

Constant Field Values

USER_DEFAULT_SCALE

@HiveDecimalVersionV1
public static final int USER_DEFAULT_SCALE

See Also:

Constant Field Values

SYSTEM_DEFAULT_PRECISION

@HiveDecimalVersionV1
public static final int SYSTEM_DEFAULT_PRECISION

Default precision/scale when system is not able to determine them, such as in case of a non-generic udf.

See Also:

Constant Field Values

SYSTEM_DEFAULT_SCALE

@HiveDecimalVersionV1
public static final int SYSTEM_DEFAULT_SCALE

See Also:

Constant Field Values

ZERO

@HiveDecimalVersionV1
public static final HiveDecimal ZERO

Common values.

ONE

@HiveDecimalVersionV1
public static final HiveDecimal ONE

ROUND_FLOOR

@HiveDecimalVersionV1
public static final int ROUND_FLOOR

ROUND_FLOOR:

Round towards negative infinity.

The Hive function is FLOOR.

Positive numbers: The round fraction is thrown away.

(Example here rounds at scale 0) Value FLOOR 0.3 0 2 2 2.1 2

Negative numbers: If there is a round fraction, throw it away and subtract 1.

(Example here rounds at scale 0) Value FLOOR -0.3 -1 -2 -2 -2.1 -3

See Also:

Constant Field Values

ROUND_CEILING

@HiveDecimalVersionV1
public static final int ROUND_CEILING

ROUND_CEILING:

Round towards positive infinity.

The Hive function is CEILING.

Positive numbers: If there is a round fraction, throw it away and add 1

(Example here rounds at scale 0) Value CEILING 0.3 1 2 2 2.1 3

Negative numbers: The round fraction is thrown away.

(Example here rounds at scale 0) Value CEILING -0.3 0 -2 -2 -2.1 -2

See Also:

Constant Field Values

ROUND_HALF_UP

@HiveDecimalVersionV1
public static final int ROUND_HALF_UP

ROUND_HALF_UP:

Round towards "nearest neighbor" unless both neighbors are equidistant then round up.

The Hive function is ROUND.

For result, throw away round fraction. If the round fraction is >= 0.5, then add 1 when positive and subtract 1 when negative. So, the sign is irrelevant.

(Example here rounds at scale 0) Value ROUND Value ROUND 0.3 0 -0.3 0 2 2 -2 -2 2.1 2 -2.1 -2 2.49 2 -2.49 -2 2.5 3 -2.5 -3

See Also:

Constant Field Values

ROUND_HALF_EVEN

@HiveDecimalVersionV1
public static final int ROUND_HALF_EVEN

ROUND_HALF_EVEN: Round towards the "nearest neighbor" unless both neighbors are equidistant, then round towards the even neighbor.

The Hive function is BROUND.

Known as Banker’s Rounding.

When you add values rounded with ROUND_HALF_UP you have a bias that grows as you add more numbers. Banker's Rounding is a way to minimize that bias. It rounds toward the nearest even number when the fraction is 0.5 exactly. In table below, notice that 2.5 goes DOWN to 2 (even) but 3.5 goes UP to 4 (even), etc.

So, the sign is irrelevant.

(Example here rounds at scale 0) Value BROUND Value BROUND 0.49 0 -0.49 0 0.5 0 -0.5 0 0.51 1 -0.51 -1 1.5 2 -1.5 -2 2.5 2 -2.5 -2 2.51 3 -2.51 -3 3.5 4 -3.5 -4 4.5 4 -4.5 -4 4.51 5 -4.51 -5

See Also:

Constant Field Values

SCRATCH_BUFFER_LEN_SERIALIZATION_UTILS_READ

@HiveDecimalVersionV2
public static final int SCRATCH_BUFFER_LEN_SERIALIZATION_UTILS_READ

See Also:

Constant Field Values

SCRATCH_LONGS_LEN

@HiveDecimalVersionV2
public static final int SCRATCH_LONGS_LEN

See Also:

Constant Field Values

SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES

@HiveDecimalVersionV2
public static final int SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES

See Also:

Constant Field Values

SCRATCH_BUFFER_LEN_TO_BYTES

@HiveDecimalVersionV2
public static final int SCRATCH_BUFFER_LEN_TO_BYTES

See Also:

Constant Field Values

Method Detail

createFromFast

@HiveDecimalVersionV2
public static HiveDecimal createFromFast(FastHiveDecimal fastDec)

Create a HiveDecimal from a FastHiveDecimal object. Used by HiveDecimalWritable.

Parameters:

fastDec - the value to set

Returns:

new hive decimal

create

@HiveDecimalVersionV1
public static HiveDecimal create(BigDecimal bigDecimal)

Create a HiveDecimal from BigDecimal object. A BigDecimal object has a decimal scale. We will have overflow if BigDecimal's integer part exceed MAX_PRECISION digits or 99,999,999,999,999,999,999,999,999,999,999,999,999 or 10^38 - 1. When the BigDecimal value's precision exceeds MAX_PRECISION and there are fractional digits because of scale > 0, then lower digits are trimmed off with rounding to meet the MAX_PRECISION requirement. Also, BigDecimal supports negative scale -- which means multiplying the value by 10^abs(scale). And, BigDecimal allows for a non-zero scale for zero. We normalize that so zero always has scale 0.

Parameters:

bigDecimal - the value to set

Returns:

The HiveDecimal with the BigDecimal's value adjusted down to a maximum precision. Otherwise, null is returned for overflow.

create

@HiveDecimalVersionV1
public static HiveDecimal create(BigDecimal bigDecimal,
                                                       boolean allowRounding)

Same as the above create method, except fractional digit rounding can be turned off.

Parameters:

bigDecimal - the value to set

allowRounding - True requires all of the bigDecimal value be converted to the decimal without loss of precision.

Returns:

create

@HiveDecimalVersionV1
public static HiveDecimal create(BigInteger bigInteger)

Creates a HiveDecimal from a BigInteger's value with a scale of 0. We will have overflow if BigInteger exceed MAX_PRECISION digits or 99,999,999,999,999,999,999,999,999,999,999,999,999 or 10^38 - 1.

Parameters:

bigInteger - the value to set

Returns:

A HiveDecimal object with the exact BigInteger's value. Otherwise, null is returned on overflow.

create

@HiveDecimalVersionV1
public static HiveDecimal create(BigInteger bigInteger,
                                                       int scale)

Creates a HiveDecimal from a BigInteger's value with a specified scale. We will have overflow if BigInteger exceed MAX_PRECISION digits or 99,999,999,999,999,999,999,999,999,999,999,999,999 or 10^38 - 1. The resulting decimal will have fractional digits when the specified scale is greater than 0. When the BigInteger's value's precision exceeds MAX_PRECISION and there are fractional digits because of scale > 0, then lower digits are trimmed off with rounding to meet the MAX_PRECISION requirement.

Parameters:

bigInteger - the value to set

scale - the scale to set

Returns:

A HiveDecimal object with the BigInteger's value adjusted for scale. Otherwise, null is returned on overflow.

create

@HiveDecimalVersionV1
public static HiveDecimal create(String string)

Create a HiveDecimal by parsing a whole string. We support parsing a decimal with an exponent because the previous version (i.e. OldHiveDecimal) uses the BigDecimal parser and was able to.

Parameters:

string - the string to parse

Returns:

a new hive decimal

create

@HiveDecimalVersionV2
public static HiveDecimal create(String string,
                                                       boolean trimBlanks)

Same as the method above, except blanks before and after are tolerated.

Parameters:

string - the string to parse

trimBlanks - True specifies leading and trailing blanks are to be ignored.

Returns:

a new hive decimal

create

@HiveDecimalVersionV2
public static HiveDecimal create(byte[] bytes)

Create a HiveDecimal by parsing the characters in a whole byte array. Same rules as create(String string) above.

create

@HiveDecimalVersionV2
public static HiveDecimal create(byte[] bytes,
                                                       boolean trimBlanks)

Same as the method above, except blanks before and after are tolerated.

create

@HiveDecimalVersionV2
public static HiveDecimal create(boolean isNegative,
                                                       byte[] bytes,
                                                       int scale)

This method takes in digits only UTF-8 characters, a sign flag, and a scale and returns a decimal.

create

@HiveDecimalVersionV2
public static HiveDecimal create(boolean isNegative,
                                                       byte[] bytes,
                                                       int offset,
                                                       int length,
                                                       int scale)

create

@HiveDecimalVersionV2
public static HiveDecimal create(byte[] bytes,
                                                       int offset,
                                                       int length)

Create a HiveDecimal by parsing the characters in a slice of a byte array. Same rules as create(String string) above.

create

@HiveDecimalVersionV2
public static HiveDecimal create(byte[] bytes,
                                                       int offset,
                                                       int length,
                                                       boolean trimBlanks)

Same as the method above, except blanks before and after are tolerated.

create

@HiveDecimalVersionV1
public static HiveDecimal create(int intValue)

Create a HiveDecimal object from an int.

create

@HiveDecimalVersionV1
public static HiveDecimal create(long longValue)

Create a HiveDecimal object from a long.

create

@HiveDecimalVersionV2
public static HiveDecimal create(long longValue,
                                                       int scale)

Create a HiveDecimal object from a long with a specified scale.

create

@HiveDecimalVersionV2
public static HiveDecimal create(float floatValue)

Create a HiveDecimal object from a float.

This method is equivalent to HiveDecimal.create(Float.toString(floatValue))

create

@HiveDecimalVersionV2
public static HiveDecimal create(double doubleValue)

Create a HiveDecimal object from a double.

This method is equivalent to HiveDecimal.create(Double.toString(doubleValue))

serializationUtilsRead

@HiveDecimalVersionV2
public static HiveDecimal serializationUtilsRead(InputStream inputStream,
                                                                       int scale,
                                                                       byte[] scratchBytes)
                                                                throws IOException

Deserialize data written in the format used by the SerializationUtils methods readBigInteger/writeBigInteger and create a decimal using the supplied scale.

ORC uses those SerializationUtils methods for its serialization.

A scratch bytes array is necessary to do the binary to decimal conversion for better performance. Pass a SCRATCH_BUFFER_LEN_SERIALIZATION_UTILS_READ byte array for scratchBytes.

Returns:

The deserialized decimal or null if the conversion failed.

Throws:

IOException

createFromBigIntegerBytesAndScale

@HiveDecimalVersionV2
public static HiveDecimal createFromBigIntegerBytesAndScale(byte[] bytes,
                                                                                  int scale)

Convert bytes in the format used by BigInteger's toByteArray format (and accepted by its constructor) into a decimal using the specified scale.

Our bigIntegerBytes methods create bytes in this format, too.

This method is designed for high performance and does not create an actual BigInteger during binary to decimal conversion.

createFromBigIntegerBytesAndScale

@HiveDecimalVersionV2
public static HiveDecimal createFromBigIntegerBytesAndScale(byte[] bytes,
                                                                                  int offset,
                                                                                  int length,
                                                                                  int scale)

serializationUtilsWrite

@HiveDecimalVersionV2
public boolean serializationUtilsWrite(OutputStream outputStream,
                                                             long[] scratchLongs)
                                                      throws IOException

Serialize this decimal's BigInteger equivalent unscaled value using the format that the SerializationUtils methods readBigInteger/writeBigInteger use.

ORC uses those SerializationUtils methods for its serialization.

Scratch objects necessary to do the decimal to binary conversion without actually creating a BigInteger object are passed for better performance.

Allocate scratchLongs with SCRATCH_LONGS_LEN longs.

Throws:

IOException

bigIntegerBytes

@HiveDecimalVersionV2
public int bigIntegerBytes(long[] scratchLongs,
                                                 byte[] buffer)

Return binary representation of this decimal's BigInteger equivalent unscaled value using the format that the BigInteger's toByteArray method returns (and the BigInteger constructor accepts).

Used by LazyBinary, Avro, and Parquet serialization.

Scratch objects necessary to do the decimal to binary conversion without actually creating a BigInteger object are passed for better performance.

Allocate scratchLongs with SCRATCH_LONGS_LEN longs. And, allocate buffer with SCRATCH_BUFFER_LEN_BIG_INTEGER_BYTES bytes.

Parameters:

scratchLongs -

buffer -

Returns:

The number of bytes used for the binary result in buffer. Otherwise, 0 if the conversion failed.

bigIntegerBytes

@HiveDecimalVersionV2
public byte[] bigIntegerBytes()

bigIntegerBytesScaled

@HiveDecimalVersionV2
public int bigIntegerBytesScaled(int serializeScale,
                                                       long[] scratchLongs,
                                                       byte[] buffer)

Convert decimal to BigInteger binary bytes with a serialize scale, similar to the formatScale for toFormatString. It adds trailing zeroes the (emulated) BigInteger toByteArray result when a serializeScale is greater than current scale. Or, rounds if scale is less than current scale.

Used by Avro and Parquet serialization.

This emulates the OldHiveDecimal setScale AND THEN OldHiveDecimal getInternalStorage() behavior.

bigIntegerBytesScaled

@HiveDecimalVersionV2
public byte[] bigIntegerBytesScaled(int serializeScale)

toString

@HiveDecimalVersionV1
public String toString()

Return a string representation of the decimal.

It is the equivalent of calling bigDecimalValue().toPlainString -- it does not add exponent notation -- but is much faster.

NOTE: If setScale(int serializationScale) was used to create the decimal object, then trailing fractional digits will be added to display to the serializationScale. Or, the display may get rounded. See the comments for that method.

Overrides:

toString in class FastHiveDecimal

toString

@HiveDecimalVersionV2
public String toString(byte[] scratchBuffer)

toFormatString

@HiveDecimalVersionV1
public String toFormatString(int formatScale)

Return a string representation of the decimal using the specified scale.

This method is designed to ALWAYS SUCCEED (unless the newScale parameter is out of range).

Is does the equivalent of a setScale(int newScale). So, more than 38 digits may be returned. See that method for more details on how this can happen.

Parameters:

formatScale - The number of digits after the decimal point

Returns:

The scaled decimal representation string representation.

toFormatString

@HiveDecimalVersionV2
public String toFormatString(int formatScale,
                                                   byte[] scratchBuffer)

toDigitsOnlyString

@HiveDecimalVersionV2
public String toDigitsOnlyString()

toBytes

@HiveDecimalVersionV2
public int toBytes(byte[] scratchBuffer)

Decimal to ASCII bytes conversion.

The scratch buffer will contain the result afterwards. It should be SCRATCH_BUFFER_LEN_TO_BYTES bytes long.

The result is produced at the end of the scratch buffer, so the return value is the byte index of the first byte. The byte slice is [byteIndex:SCRATCH_BUFFER_LEN_TO_BYTES-1].

toFormatBytes

@HiveDecimalVersionV2
public int toFormatBytes(int formatScale,
                                               byte[] scratchBuffer)

This is the serialization version of decimal to string conversion.

It adds trailing zeroes when the formatScale is greater than the current scale. Or, it does round if the formatScale is less than the current scale.

Note that you can get more than 38 (MAX_PRECISION) digits in the output with this method.

toDigitsOnlyBytes

@HiveDecimalVersionV2
public int toDigitsOnlyBytes(byte[] scratchBuffer)

Convert decimal to just the digits -- no dot.

Currently used by BinarySortable serialization.

A faster way to get just the digits than calling unscaledValue.toString().getBytes().

compareTo

@HiveDecimalVersionV1
public int compareTo(HiveDecimal dec)

Specified by:

compareTo in interface Comparable<HiveDecimal>

newFasterHashCode

@HiveDecimalVersionV2
public int newFasterHashCode()

Hash code based on (new) decimal representation.

Faster than hashCode().

Used by map join and other Hive internal purposes where performance is important.

IMPORTANT: See comments for hashCode(), too.

hashCode

@HiveDecimalVersionV1
public int hashCode()

This is returns original hash code as returned by HiveDecimalV1.

We need this when the HiveDecimalV1 hash code has been exposed and and written or affected how data is written.

This method supports compatibility.

Examples: bucketing, Hive hash() function, and Hive statistics.

NOTE: It is necessary to create a BigDecimal object and use its hash code, so this method is slow.

Overrides:

hashCode in class Object

equals

@HiveDecimalVersionV1
public boolean equals(Object obj)

Are two decimal content (values) equal?

Overrides:

equals in class Object

Parameters:

obj - The 2nd decimal.

Returns:

When obj is null or not class HiveDecimal, the return is false. Otherwise, returns true when the decimal values are exactly equal.

scale

@HiveDecimalVersionV1
public int scale()

Returns the scale of the decimal. Range 0 .. MAX_SCALE.

integerDigitCount

@HiveDecimalVersionV2
public int integerDigitCount()

Returns the number of integer digits in the decimal.

When the integer portion is zero, this method returns 0.

precision

@HiveDecimalVersionV1
public int precision()

Returns the number of digits (integer and fractional) in the number, which is equivalent to SQL decimal precision.

Note that this method is different from rawPrecision(), which returns the number of digits ignoring the scale. Note that rawPrecision returns 0 when the value is 0. Decimal precision rawPrecision 0 1 0 1 1 1 -7 1 1 0.1 1 1 0.04 2 1 0.00380 5 3 104.0009 7 7

If you just want the actual number of digits, use rawPrecision().

rawPrecision

@HiveDecimalVersionV2
public int rawPrecision()

signum

@HiveDecimalVersionV1
public int signum()

Get the sign of the decimal.

Returns:

0 if the decimal is equal to 0, -1 if less than zero, and 1 if greater than 0

isByte

@HiveDecimalVersionV2
public boolean isByte()

Is the decimal value a byte? Range -128 to 127. Byte.MIN_VALUE Byte.MAX_VALUE

Emulates testing for no value corruption: bigDecimalValue().setScale(0).equals(BigDecimal.valueOf(bigDecimalValue().byteValue()))

NOTE: Fractional digits are ignored in the test since byteValue() will remove them (round down).

Returns:

True when byteValue() will return a correct byte.

byteValue

@HiveDecimalVersionV1
public byte byteValue()

A byte variation of longValue()

This method will return a corrupted value unless isByte() is true.

isShort

@HiveDecimalVersionV2
public boolean isShort()

Is the decimal value a short? Range -32,768 to 32,767. Short.MIN_VALUE Short.MAX_VALUE

Emulates testing for no value corruption: bigDecimalValue().setScale(0).equals(BigDecimal.valueOf(bigDecimalValue().shortValue()))

NOTE: Fractional digits are ignored in the test since shortValue() will remove them (round down).

Returns:

True when shortValue() will return a correct short.

shortValue

@HiveDecimalVersionV1
public short shortValue()

A short variation of longValue().

This method will return a corrupted value unless isShort() is true.

isInt

@HiveDecimalVersionV2
public boolean isInt()

Is the decimal value a int? Range -2,147,483,648 to 2,147,483,647. Integer.MIN_VALUE Integer.MAX_VALUE

Emulates testing for no value corruption: bigDecimalValue().setScale(0).equals(BigDecimal.valueOf(bigDecimalValue().intValue()))

NOTE: Fractional digits are ignored in the test since intValue() will remove them (round down).

Returns:

True when intValue() will return a correct int.

intValue

@HiveDecimalVersionV1
public int intValue()

An int variation of longValue().

This method will return a corrupted value unless isInt() is true.

isLong

@HiveDecimalVersionV2
public boolean isLong()

Is the decimal value a long? Range -9,223,372,036,854,775,808 to 9,223,372,036,854,775,807. Long.MIN_VALUE Long.MAX_VALUE

Emulates testing for no value corruption: bigDecimalValue().setScale(0).equals(BigDecimal.valueOf(bigDecimalValue().longValue()))

NOTE: Fractional digits are ignored in the test since longValue() will remove them (round down).

Returns:

True when longValue() will return a correct long.

longValue

@HiveDecimalVersionV1
public long longValue()

Return the long value of a decimal.

This method will return a corrupted value unless isLong() is true.

longValueExact

@HiveDecimalVersionV1
public long longValueExact()

floatValue

@HiveDecimalVersionV1
public float floatValue()

Return a float representing the decimal. Due the limitations of float, some values will not be accurate.

doubleValue

@HiveDecimalVersionV1
public double doubleValue()

Return a double representing the decimal. Due the limitations of double, some values will not be accurate.

bigDecimalValue

@HiveDecimalVersionV1
public BigDecimal bigDecimalValue()

Return a BigDecimal representing the decimal. The BigDecimal class is able to accurately represent the decimal. NOTE: We are not representing our decimal as BigDecimal now as OldHiveDecimal did, so this is now slower.

unscaledValue

@HiveDecimalVersionV1
public BigInteger unscaledValue()

Get a BigInteger representing the decimal's digits without a dot.

Returns:

Returns a signed BigInteger.

fractionPortion

@HiveDecimalVersionV2
public HiveDecimal fractionPortion()

Return a decimal with only the fractional digits.

Zero is returned when there are no fractional digits (i.e. scale is 0).

integerPortion

@HiveDecimalVersionV2
public HiveDecimal integerPortion()

Return a decimal with only the integer digits.

Any fractional digits are removed. E.g. 2.083 scale 3 returns as 2 scale 0.

add

@HiveDecimalVersionV1
public HiveDecimal add(HiveDecimal dec)

Add the current decimal and another decimal and return the result.

subtract

@HiveDecimalVersionV1
public HiveDecimal subtract(HiveDecimal dec)

Subtract from the current decimal another decimal and return the result.

multiply

@HiveDecimalVersionV1
public HiveDecimal multiply(HiveDecimal dec)

Multiply two decimals.

NOTE: Overflow Determination for Multiply

OldDecimal.multiply performs the multiply with BigDecimal but DOES NOT ALLOW ROUNDING (i.e. no throwing away lower fractional digits).

CONSIDER: Allowing rounding. This would eliminate cases today where we return null for the multiplication result.

IMPLEMENTATION NOTE: HiveDecimalV1 code does this:

return create(bd.multiply(dec.bd), false);

scaleByPowerOfTen

@HiveDecimalVersionV1
public HiveDecimal scaleByPowerOfTen(int power)

Multiplies a decimal by a power of 10.

The decimal 19350 scale 0 will return 193.5 scale 1 when power is -2 (negative).

The decimal 1.000923 scale 6 will return 10009.23 scale 2 when power is 4 (positive).

Parameters:

power -

Returns:

Returns a HiveDecimal whose value is value * 10^power.

abs

@HiveDecimalVersionV1
public HiveDecimal abs()

Take the absolute value of a decimal.

Returns:

When the decimal is negative, returns a new HiveDecimal with the positive value. Otherwise, returns the current 0 or positive value object;

negate

@HiveDecimalVersionV1
public HiveDecimal negate()

Reverse the sign of a decimal.

Returns:

Returns a new decimal with the sign flipped. When the value is 0, the current object is returned.

setScale

@Deprecated
 @HiveDecimalVersionV1
public HiveDecimal setScale(int serializationScale)

Deprecated.

DEPRECATED for V2.

Create a decimal from another decimal whose only change is it is MARKED and will display / serialize with a specified scale that will add trailing zeroes (or round) if necessary.

After display / serialization, the MARKED object is typically thrown away.

A MARKED decimal ONLY affects these 2 methods since these were the only ways setScale was used in the old code.

toString unscaleValue

This method has been deprecated because has poor performance by creating a throw away object.

For setScale(scale).toString() use toFormatString(scale) instead. For setScale(scale).unscaledValue().toByteArray() use V2 bigIntegerBytesScaled(scale) instead.

For better performance, use the V2 form of toFormatString that takes a scratch buffer, or even better use toFormatBytes.

And, use the form of bigIntegerBytesScaled that takes scratch objects for better performance.

setScale

@HiveDecimalVersionV1
public HiveDecimal setScale(int roundingPoint,
                                                  int roundingMode)

Do decimal rounding and return the result.

When the roundingPoint is 0 or positive, we round away lower fractional digits if the roundingPoint is less than current scale. In this case, we will round the result using the specified rounding mode.

When the roundingPoint is negative, the rounding will occur within the integer digits. Integer digits below the roundPoint will be cleared. If the rounding occurred, a one will be added just above the roundingPoint. Note this may cause overflow.

No effect when the roundingPoint equals the current scale. The current object is returned.

The name setScale is taken from BigDecimal.setScale -- a better name would have been round.

pow

@HiveDecimalVersionV1
public HiveDecimal pow(int exponent)

Return the result of decimal^exponent

CONSIDER: Currently, negative exponent is not supported. CONSIDER: Does anybody use this method?

divide

@HiveDecimalVersionV1
public HiveDecimal divide(HiveDecimal divisor)

Divides this decimal by another decimal and returns a new decimal with the result.

remainder

@HiveDecimalVersionV1
public HiveDecimal remainder(HiveDecimal divisor)

Divides this decimal by another decimal and returns a new decimal with the remainder of the division.

value is (decimal % divisor)

The remainder is equivalent to BigDecimal: bigDecimalValue().subtract(bigDecimalValue().divideToIntegralValue(divisor).multiply(divisor))

enforcePrecisionScale

@HiveDecimalVersionV1
public static HiveDecimal enforcePrecisionScale(HiveDecimal dec,
                                                                      int maxPrecision,
                                                                      int maxScale)

Determine if a decimal fits within a specified maxPrecision and maxScale, and round off fractional digits if necessary to make the decimal fit.

The relationship between the enforcement maxPrecision and maxScale is restricted. The specified maxScale must be less than or equal to the maxPrecision.

Normally, decimals that result from creation operation, arithmetic operations, etc are "free range" up to MAX_PRECISION and MAX_SCALE. Each operation checks if the result decimal is beyond MAX_PRECISION and MAX_SCALE. If so the result decimal is rounded off using ROUND_HALF_UP. If the round digit is 5 or more, one is added to the lowest remaining digit. The round digit is the digit just below the round point. Result overflow can occur if a result decimal's integer portion exceeds MAX_PRECISION.

This method supports enforcing to a declared Hive DECIMAL's precision/scale. E.g. DECIMAL(10,4)

Here are the enforcement/rounding checks of this method:

1) Maximum integer digits = maxPrecision - maxScale

If the decimal's integer digit count exceeds this, the decimal does not fit (overflow).

2) If decimal's scale is greater than maxScale, then excess fractional digits are rounded off. When rounding increases the remaining decimal, it may exceed the limits and overflow.

Parameters:

dec -

maxPrecision -

maxScale -

Returns:

The original decimal if no adjustment is necessary. A rounded off decimal if adjustment was necessary. Otherwise, null if the decimal doesn't fit within maxPrecision / maxScale or rounding caused a result that exceeds the specified limits or MAX_PRECISION integer digits.

validate

@HiveDecimalVersionV2
public void validate()

Throws an exception if the current decimal value is invalid.