org.apache.hadoop.hive.ql.exec.vector.reducesink

Class VectorReduceSinkObjectHashOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<T>

org.apache.hadoop.hive.ql.exec.TerminalOperator<ReduceSinkDesc>

org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkCommonOperator

org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkObjectHashOperator

All Implemented Interfaces:

Serializable, Cloneable, TopNHash.BinaryCollector, VectorizationContextRegion, VectorizationOperator, Node

public class VectorReduceSinkObjectHashOperator
extends VectorReduceSinkCommonOperator

This class is the object hash (not Uniform Hash) operator class for native vectorized reduce sink. It takes the "object" hash code of bucket and/or partition keys (which are often subsets of the reduce key). If the bucket and partition keys are empty, the hash will be a random number.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	isEmptyBuckets
protected boolean	isEmptyPartitions
protected ByteStream.Output	keyOutput
protected VectorSerializeRow<BinarySortableSerializeWrite>	keyVectorSerializeRow
protected int[]	reduceSinkBucketColumnMap
protected VectorExpression[]	reduceSinkBucketExpressions
protected TypeInfo[]	reduceSinkBucketTypeInfos
protected int[]	reduceSinkPartitionColumnMap
protected VectorExpression[]	reduceSinkPartitionExpressions
protected TypeInfo[]	reduceSinkPartitionTypeInfos

Fields inherited from class org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkCommonOperator

batchCounter, isEmptyKey, isEmptyValue, keyBinarySortableSerializeWrite, keyWritable, reducerHash, reduceSinkKeyColumnMap, reduceSinkKeyExpressions, reduceSinkKeyTypeInfos, reduceSinkValueColumnMap, reduceSinkValueExpressions, reduceSinkValueTypeInfos, reduceSkipTag, reduceTagByte, taskName, valueBytesWritable, valueLazyBinarySerializeWrite, valueOutput, valueVectorSerializeRow, vContext, vectorDesc, vectorReduceSinkInfo

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, alias, asyncInitOperations, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, inputObjInspectors, numRows, operatorId, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	VectorReduceSinkObjectHashOperator() Kryo ctor.
	VectorReduceSinkObjectHashOperator(CompilationOpContext ctx)
	VectorReduceSinkObjectHashOperator(CompilationOpContext ctx, OperatorDesc conf, VectorizationContext vContext, VectorDesc vectorDesc)

Method Summary

Modifier and Type	Method and Description
protected void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
void	process(Object row, int tag) Process the row.

Methods inherited from class org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkCommonOperator

closeOp, collect, collect, getInputVectorizationContext, getIsReduceSink, getName, getOperatorName, getOutputVectorizationContext, getReduceOutputName, getType, getVectorDesc, initializeEmptyKey, setOutputCollector

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, batchNeedsClone, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, endGroup, flush, flushRecursive, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEquals, logicalEqualsTree, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, processGroup, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, replaceTabAlias, reset, setAlias, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setInputContext, setInputObjInspectors, setMarker, setNextVectorBatchGroupStatus, setOpTraits, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString, toString, vectorForward

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

isEmptyBuckets

protected boolean isEmptyBuckets

reduceSinkBucketColumnMap

protected int[] reduceSinkBucketColumnMap

reduceSinkBucketTypeInfos

protected TypeInfo[] reduceSinkBucketTypeInfos

reduceSinkBucketExpressions

protected VectorExpression[] reduceSinkBucketExpressions

isEmptyPartitions

protected boolean isEmptyPartitions

reduceSinkPartitionColumnMap

protected int[] reduceSinkPartitionColumnMap

reduceSinkPartitionTypeInfos

protected TypeInfo[] reduceSinkPartitionTypeInfos

reduceSinkPartitionExpressions

protected VectorExpression[] reduceSinkPartitionExpressions

keyOutput

protected transient ByteStream.Output keyOutput

keyVectorSerializeRow

protected transient VectorSerializeRow<BinarySortableSerializeWrite> keyVectorSerializeRow

Constructor Detail

VectorReduceSinkObjectHashOperator

protected VectorReduceSinkObjectHashOperator()

Kryo ctor.

VectorReduceSinkObjectHashOperator

public VectorReduceSinkObjectHashOperator(CompilationOpContext ctx)

VectorReduceSinkObjectHashOperator

public VectorReduceSinkObjectHashOperator(CompilationOpContext ctx,
                                          OperatorDesc conf,
                                          VectorizationContext vContext,
                                          VectorDesc vectorDesc)
                                   throws HiveException

Throws:

HiveException

Method Detail

initializeOp

protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class VectorReduceSinkCommonOperator

Throws:

HiveException

process

public void process(Object row,
                    int tag)
             throws HiveException

Description copied from class: Operator

Process the row.

Specified by:

process in class Operator<ReduceSinkDesc>

Parameters:

row - The object representing the row.

tag - The tag of the row usually means which parent this row comes from. Rows with the same tag should have exactly the same rowInspector all the time.

Throws:

HiveException