org.apache.hadoop.hive.ql.exec.vector.ptf

Class VectorPTFOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<PTFDesc>

org.apache.hadoop.hive.ql.exec.vector.ptf.VectorPTFOperator

All Implemented Interfaces:

Serializable, Cloneable, VectorizationContextRegion, VectorizationOperator, Node

public class VectorPTFOperator
extends Operator<PTFDesc>
implements VectorizationOperator, VectorizationContextRegion

This class is native vectorized PTF operator class.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected VectorizationContext	vOutContext

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, alias, asyncInitOperations, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, inputObjInspectors, numRows, operatorId, out, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	VectorPTFOperator() Kryo ctor.
	VectorPTFOperator(CompilationOpContext ctx)
	VectorPTFOperator(CompilationOpContext ctx, OperatorDesc conf, VectorizationContext vContext, VectorDesc vectorDesc)

Method Summary

Modifier and Type	Method and Description
protected void	closeOp(boolean abort) Operator specific close routine.
VectorizationContext	getInputVectorizationContext()
String	getName()
static String	getOperatorName()
VectorizationContext	getOutputVectorizationContext()
OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
VectorDesc	getVectorDesc()
protected void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
void	process(Object row, int tag) We are processing a batch from reduce processor that is only for one reducer key or PTF group.
void	setNextVectorBatchGroupStatus(boolean isLastGroupBatch)
protected void	setupVOutContext() Setup the vectorized row batch description of the output of the native vectorized PTF operator.

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, endGroup, flush, flushRecursive, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getIsReduceSink, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getReduceOutputName, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEquals, logicalEqualsTree, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, processGroup, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, replaceTabAlias, reset, setAlias, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setInputContext, setInputObjInspectors, setMarker, setOpTraits, setOutputCollector, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString, toString, vectorForward

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

vOutContext

protected VectorizationContext vOutContext

Constructor Detail

VectorPTFOperator

protected VectorPTFOperator()

Kryo ctor.

VectorPTFOperator

public VectorPTFOperator(CompilationOpContext ctx)

VectorPTFOperator

public VectorPTFOperator(CompilationOpContext ctx,
                         OperatorDesc conf,
                         VectorizationContext vContext,
                         VectorDesc vectorDesc)
                  throws HiveException

Throws:

HiveException

Method Detail

setupVOutContext

protected void setupVOutContext()

Setup the vectorized row batch description of the output of the native vectorized PTF operator. Use the output projection we previously built from a mixture of input columns and new scratch columns.

initializeOp

protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<PTFDesc>

Throws:

HiveException

setNextVectorBatchGroupStatus

public void setNextVectorBatchGroupStatus(boolean isLastGroupBatch)
                                   throws HiveException

Overrides:

setNextVectorBatchGroupStatus in class Operator<PTFDesc>

Throws:

HiveException

process

public void process(Object row,
                    int tag)
             throws HiveException

We are processing a batch from reduce processor that is only for one reducer key or PTF group. For a simple OVER (PARTITION BY column) or OVER (ORDER BY column), the reduce processor's group key is the partition or order by key. For an OVER (PARTITION BY column1, ORDER BY column2), the reduce-shuffle group key is the combination of the partition column1 and the order by column2. In this case, this method has to watch for changes in the partition and reset the group aggregations. The reduce processor calls setNextVectorBatchGroupStatus beforehand to tell us whether the batch supplied to our process method is the last batch for the group key, or not. This helps us intelligently process the batch.

Specified by:

process in class Operator<PTFDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException

closeOp

protected void closeOp(boolean abort)
                throws HiveException

Description copied from class: Operator

Operator specific close routine. Operators which inherents this class should overwrite this funtion for their specific cleanup routine.

Overrides:

closeOp in class Operator<PTFDesc>

Throws:

HiveException

getName

public String getName()

Specified by:

getName in interface Node

Specified by:

getName in class Operator<PTFDesc>

Returns:

the name of the operator

getOperatorName

public static String getOperatorName()

getType

public OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<PTFDesc>

Returns:

OperatorType.*

getOutputVectorizationContext

public VectorizationContext getOutputVectorizationContext()

Specified by:

getOutputVectorizationContext in interface VectorizationContextRegion

getInputVectorizationContext

public VectorizationContext getInputVectorizationContext()

Specified by:

getInputVectorizationContext in interface VectorizationOperator

getVectorDesc

public VectorDesc getVectorDesc()

Specified by:

getVectorDesc in interface VectorizationOperator