org.apache.hadoop.hive.ql.udf.generic

Class GenericUDAFPercentileCont.PercentileContEvaluator<T,U>

java.lang.Object

org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont.PercentileContEvaluator<T,U>

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

GenericUDAFPercentileCont.PercentileContDoubleEvaluator, GenericUDAFPercentileCont.PercentileContLongEvaluator

Enclosing class:

GenericUDAFPercentileCont

public abstract static class GenericUDAFPercentileCont.PercentileContEvaluator<T,U>
extends GenericUDAFEvaluator

An abstract class to hold the generic udf functions for calculating percentile.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	GenericUDAFPercentileCont.PercentileContEvaluator.PercentileAgg A state class to store intermediate aggregation results.

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

GenericUDAFEvaluator.AbstractAggregationBuffer, GenericUDAFEvaluator.AggregationBuffer, GenericUDAFEvaluator.AggregationType, GenericUDAFEvaluator.Mode

Field Summary

Fields

Modifier and Type	Field and Description
protected StructField	countsField
protected PrimitiveObjectInspector	inputOI
protected boolean	isAscending
protected StructField	isAscendingField
protected Object[]	partialResult
protected StructField	percentilesField
protected List<DoubleWritable>	results
protected StructObjectInspector	soi

Fields inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

partitionEvaluator

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	PercentileContEvaluator(Comparator<Map.Entry<U,org.apache.hadoop.io.LongWritable>> comparator, org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont.Converter converter)

Method Summary

Modifier and Type	Method and Description
protected List<DoubleWritable>	calculatePercentile(List<DoubleWritable> percentiles, List<Map.Entry<U,org.apache.hadoop.io.LongWritable>> entriesList, long total, List<DoubleWritable> results)
protected abstract U	copyInput(U input)
protected abstract GenericUDAFPercentileCont.PercentileCalculator<U>	getCalculator()
protected abstract T	getInput(Object object, PrimitiveObjectInspector inputOI)
GenericUDAFEvaluator.AggregationBuffer	getNewAggregationBuffer() Get a new aggregation object.
protected abstract ArrayList<ObjectInspector>	getPartialInspectors()
protected long	getTotal(List<Map.Entry<U,org.apache.hadoop.io.LongWritable>> entriesList)
protected void	increment(GenericUDAFPercentileCont.PercentileContEvaluator.PercentileAgg s, U input, long i)
ObjectInspector	init(GenericUDAFEvaluator.Mode m, ObjectInspector[] parameters) Initialize the evaluator.
protected void	initInspectors(PrimitiveObjectInspector orderByColumnOI)
protected void	initInspectors(PrimitiveObjectInspector orderByColumnOI, WritableConstantIntObjectInspector orderDirectionOI)
protected void	initPartialInspectors(StructObjectInspector objectInspector)
void	iterate(GenericUDAFEvaluator.AggregationBuffer agg, Object[] parameters) Iterate through original data.
void	merge(GenericUDAFEvaluator.AggregationBuffer agg, Object partial) Merge with partial aggregation result.
void	reset(GenericUDAFEvaluator.AggregationBuffer agg) Reset the aggregation.
Object	terminate(GenericUDAFEvaluator.AggregationBuffer agg) Get final aggregation result.
Object	terminatePartial(GenericUDAFEvaluator.AggregationBuffer agg) Get partial aggregation result.
static void	validatePercentile(Double percentile)
protected abstract U	wrapInput(T input)

Methods inherited from class org.apache.hadoop.hive.ql.udf.generic.GenericUDAFEvaluator

aggregate, close, configure, createPartitionEvaluator, estimate, evaluate, getExprString, getPartitionWindowingEvaluator, getWindowingEvaluator, isEstimable

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inputOI

protected PrimitiveObjectInspector inputOI

partialResult

protected transient Object[] partialResult

results

protected List<DoubleWritable> results

soi

protected transient StructObjectInspector soi

countsField

protected transient StructField countsField

percentilesField

protected transient StructField percentilesField

isAscendingField

protected transient StructField isAscendingField

isAscending

protected transient boolean isAscending

Constructor Detail

PercentileContEvaluator

protected PercentileContEvaluator(Comparator<Map.Entry<U,org.apache.hadoop.io.LongWritable>> comparator,
                                  org.apache.hadoop.hive.ql.udf.generic.GenericUDAFPercentileCont.Converter converter)

Method Detail

init

public ObjectInspector init(GenericUDAFEvaluator.Mode m,
                            ObjectInspector[] parameters)
                     throws HiveException

Description copied from class: GenericUDAFEvaluator

Initialize the evaluator.

Overrides:

init in class GenericUDAFEvaluator

Parameters:

m - The mode of aggregation.

parameters - The ObjectInspector for the parameters: In PARTIAL1 and COMPLETE mode, the parameters are original data; In PARTIAL2 and FINAL mode, the parameters are just partial aggregations (in that case, the array will always have a single element).

Returns:

The ObjectInspector for the return value. In PARTIAL1 and PARTIAL2 mode, the ObjectInspector for the return value of terminatePartial() call; In FINAL and COMPLETE mode, the ObjectInspector for the return value of terminate() call. NOTE: We need ObjectInspector[] (in addition to the TypeInfo[] in GenericUDAFResolver) for 2 reasons: 1. ObjectInspector contains more information than TypeInfo; and GenericUDAFEvaluator.init at execution time. 2. We call GenericUDAFResolver.getEvaluator at compilation time,

Throws:

HiveException

getCalculator

protected abstract GenericUDAFPercentileCont.PercentileCalculator<U> getCalculator()

getPartialInspectors

protected abstract ArrayList<ObjectInspector> getPartialInspectors()

getInput

protected abstract T getInput(Object object,
                              PrimitiveObjectInspector inputOI)

wrapInput

protected abstract U wrapInput(T input)

copyInput

protected abstract U copyInput(U input)

initInspectors

protected void initInspectors(PrimitiveObjectInspector orderByColumnOI)

initInspectors

protected void initInspectors(PrimitiveObjectInspector orderByColumnOI,
                              WritableConstantIntObjectInspector orderDirectionOI)

initPartialInspectors

protected void initPartialInspectors(StructObjectInspector objectInspector)

getNewAggregationBuffer

public GenericUDAFEvaluator.AggregationBuffer getNewAggregationBuffer()
                                                               throws HiveException

Description copied from class: GenericUDAFEvaluator

Get a new aggregation object.

Specified by:

getNewAggregationBuffer in class GenericUDAFEvaluator

Throws:

HiveException

reset

public void reset(GenericUDAFEvaluator.AggregationBuffer agg)
           throws HiveException

Description copied from class: GenericUDAFEvaluator

Reset the aggregation. This is useful if we want to reuse the same aggregation.

Specified by:

reset in class GenericUDAFEvaluator

Throws:

HiveException

iterate

public void iterate(GenericUDAFEvaluator.AggregationBuffer agg,
                    Object[] parameters)
             throws HiveException

Description copied from class: GenericUDAFEvaluator

Iterate through original data.

Specified by:

iterate in class GenericUDAFEvaluator

parameters - The objects of parameters.

Throws:

HiveException

increment

protected void increment(GenericUDAFPercentileCont.PercentileContEvaluator.PercentileAgg s,
                         U input,
                         long i)

merge

public void merge(GenericUDAFEvaluator.AggregationBuffer agg,
                  Object partial)
           throws HiveException

Description copied from class: GenericUDAFEvaluator

Merge with partial aggregation result. NOTE: null might be passed in case there is no input data.

Specified by:

merge in class GenericUDAFEvaluator

partial - The partial aggregation result.

Throws:

HiveException

terminate

public Object terminate(GenericUDAFEvaluator.AggregationBuffer agg)
                 throws HiveException

Description copied from class: GenericUDAFEvaluator

Get final aggregation result.

Specified by:

terminate in class GenericUDAFEvaluator

Returns:

final aggregation result.

Throws:

HiveException

terminatePartial

public Object terminatePartial(GenericUDAFEvaluator.AggregationBuffer agg)
                        throws HiveException

Description copied from class: GenericUDAFEvaluator

Get partial aggregation result.

Specified by:

terminatePartial in class GenericUDAFEvaluator

Returns:

partial aggregation result.

Throws:

HiveException

getTotal

protected long getTotal(List<Map.Entry<U,org.apache.hadoop.io.LongWritable>> entriesList)

validatePercentile

public static void validatePercentile(Double percentile)

calculatePercentile

protected List<DoubleWritable> calculatePercentile(List<DoubleWritable> percentiles,
                                                   List<Map.Entry<U,org.apache.hadoop.io.LongWritable>> entriesList,
                                                   long total,
                                                   List<DoubleWritable> results)