org.apache.hadoop.hive.ql.exec.vector.ptf

Class VectorPTFEvaluatorBase

java.lang.Object

org.apache.hadoop.hive.ql.exec.vector.ptf.VectorPTFEvaluatorBase

Direct Known Subclasses:

VectorPTFEvaluatorAbstractAvg, VectorPTFEvaluatorAbstractLeadLag, VectorPTFEvaluatorAbstractSum, VectorPTFEvaluatorCount, VectorPTFEvaluatorCountStar, VectorPTFEvaluatorDecimalFirstValue, VectorPTFEvaluatorDecimalLastValue, VectorPTFEvaluatorDecimalMax, VectorPTFEvaluatorDecimalMin, VectorPTFEvaluatorDenseRank, VectorPTFEvaluatorDoubleFirstValue, VectorPTFEvaluatorDoubleLastValue, VectorPTFEvaluatorDoubleMax, VectorPTFEvaluatorDoubleMin, VectorPTFEvaluatorLongFirstValue, VectorPTFEvaluatorLongLastValue, VectorPTFEvaluatorLongMax, VectorPTFEvaluatorLongMin, VectorPTFEvaluatorRank, VectorPTFEvaluatorRowNumber, VectorPTFEvaluatorStreamingDecimalAvg, VectorPTFEvaluatorStreamingDecimalMax, VectorPTFEvaluatorStreamingDecimalMin, VectorPTFEvaluatorStreamingDecimalSum, VectorPTFEvaluatorStreamingDoubleAvg, VectorPTFEvaluatorStreamingDoubleMax, VectorPTFEvaluatorStreamingDoubleMin, VectorPTFEvaluatorStreamingDoubleSum, VectorPTFEvaluatorStreamingLongAvg, VectorPTFEvaluatorStreamingLongMax, VectorPTFEvaluatorStreamingLongMin, VectorPTFEvaluatorStreamingLongSum

public abstract class VectorPTFEvaluatorBase
extends Object

This is the vector PTF evaluator base class. An evaluator does the group batch aggregation work on an aggregation's 0 or 1 argument(s) and at some point will fill in an output column with the aggregation result. The aggregation argument is an input column or expression, or no argument. When the aggregation is streaming (e.g. row_number, rank, first_value, etc), the output column can be filled in immediately by the implementation of evaluateGroupBatch. For non-streaming aggregations, the aggregation result is not known until the last group batch is processed. After the last group batch has been processed, the VectorPTFGroupBatches class will call the isGroupResultNull, getResultColumnVectorType, getLongGroupResult | getDoubleGroupResult | getDecimalGroupResult, and getOutputColumnNum methods to get aggregation result information necessary to write it into the output column (as a repeated column) of all the group batches.

Field Summary

Fields

Modifier and Type	Field and Description
protected int	inputColumnNum
protected org.slf4j.Logger	LOG
protected int	outputColumnNum

Constructor Summary

Constructors

Constructor and Description
VectorPTFEvaluatorBase(WindowFrameDef windowFrameDef, int outputColumnNum)
VectorPTFEvaluatorBase(WindowFrameDef windowFrameDef, VectorExpression inputVecExpr, int outputColumnNum)

Method Summary

Modifier and Type	Method and Description
boolean	canRunOptimizedCalculation(int rowNum, Range range) Whether the evaluator can calculate the result for a given range.
void	doLastBatchWork()
abstract void	evaluateGroupBatch(VectorizedRowBatch batch)
void	evaluateInputExpr(VectorizedRowBatch batch)
Object	getGroupResult()
boolean	getNullsLast()
int	getOutputColumnNum()
abstract ColumnVector.Type	getResultColumnVectorType()
boolean	isCacheableForRange() Whether the results calculated by this evaluator are eligible for caching by PTFValueCache.
boolean	isGroupResultNull()
void	mapCustomColumns(int[] bufferedColumnMap) VectorPTFGroupBatches class works on a subset of columns, which are mapped by an array of changed indices.
void	onPartitionEnd() There might be some cleanup operations for evaluators that should not be taken care of after every row (resetEvaluator), but rather on partition-level.
void	onResultCalculated(Object result, Range range) This method is called by VectorPTFGroupBatches once the calculation is finished for a given row/range, but before the next calculation is invoked.
abstract void	resetEvaluator()
Object	runOnRange(int rowNum, Range range, VectorPTFGroupBatches batches)
void	setNullsLast(boolean nullsLast)
abstract boolean	streamsResult()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inputColumnNum

protected int inputColumnNum

outputColumnNum

protected final int outputColumnNum

LOG

protected final org.slf4j.Logger LOG

Constructor Detail

VectorPTFEvaluatorBase

public VectorPTFEvaluatorBase(WindowFrameDef windowFrameDef,
                              VectorExpression inputVecExpr,
                              int outputColumnNum)

VectorPTFEvaluatorBase

public VectorPTFEvaluatorBase(WindowFrameDef windowFrameDef,
                              int outputColumnNum)

Method Detail

getNullsLast

public boolean getNullsLast()

setNullsLast

public void setNullsLast(boolean nullsLast)

evaluateInputExpr

public void evaluateInputExpr(VectorizedRowBatch batch)
                       throws HiveException

Throws:

HiveException

evaluateGroupBatch

public abstract void evaluateGroupBatch(VectorizedRowBatch batch)
                                 throws HiveException

Throws:

HiveException

doLastBatchWork

public void doLastBatchWork()

streamsResult

public abstract boolean streamsResult()

getOutputColumnNum

public int getOutputColumnNum()

isGroupResultNull

public boolean isGroupResultNull()

getResultColumnVectorType

public abstract ColumnVector.Type getResultColumnVectorType()

getGroupResult

public Object getGroupResult()

resetEvaluator

public abstract void resetEvaluator()

canRunOptimizedCalculation

public boolean canRunOptimizedCalculation(int rowNum,
                                          Range range)

Whether the evaluator can calculate the result for a given range. The optimization is supposed to save CPU/IO cost in case of evaluators that can provide the result without iterating through the batches of a range. This method is called before every single calculation, so it lets the evaluator store state whether it's able to run an optimized calculation in a given point of time.

runOnRange

public Object runOnRange(int rowNum,
                         Range range,
                         VectorPTFGroupBatches batches)
                  throws HiveException

Throws:

HiveException

onResultCalculated

public void onResultCalculated(Object result,
                               Range range)

This method is called by VectorPTFGroupBatches once the calculation is finished for a given row/range, but before the next calculation is invoked.

Parameters:

result -

range -

onPartitionEnd

public void onPartitionEnd()

There might be some cleanup operations for evaluators that should not be taken care of after every row (resetEvaluator), but rather on partition-level.

isCacheableForRange

public boolean isCacheableForRange()

Whether the results calculated by this evaluator are eligible for caching by PTFValueCache. A result of an evaluator is cacheable, if it returns the same value for the same range within the a PTF partition, regardless of which row the current result is calculated for. That's typically true for most of the evaluators (e.g. sum), but not true for e.g. lead/lag/first_value/last_value, as their results only depend on a relation between the current and some reference row (instead of ranges).

mapCustomColumns

public void mapCustomColumns(int[] bufferedColumnMap)

VectorPTFGroupBatches class works on a subset of columns, which are mapped by an array of changed indices. Some evaluators might refer to column indices, so they can adapt to the changed column layout by this call.