org.apache.hadoop.hive.ql.exec.vector.reducesink

Class VectorReduceSinkCommonOperator

java.lang.Object

org.apache.hadoop.hive.ql.exec.Operator<T>

org.apache.hadoop.hive.ql.exec.TerminalOperator<ReduceSinkDesc>

org.apache.hadoop.hive.ql.exec.vector.reducesink.VectorReduceSinkCommonOperator

All Implemented Interfaces:

Serializable, Cloneable, TopNHash.BinaryCollector, VectorizationContextRegion, VectorizationOperator, Node

Direct Known Subclasses:

VectorReduceSinkEmptyKeyOperator, VectorReduceSinkObjectHashOperator, VectorReduceSinkUniformHashOperator

public abstract class VectorReduceSinkCommonOperator
extends TerminalOperator<ReduceSinkDesc>
implements Serializable, TopNHash.BinaryCollector, VectorizationOperator, VectorizationContextRegion

This class is common operator class for native vectorized reduce sink.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.exec.Operator

Operator.Counter, Operator.OperatorFunc, Operator.State

Field Summary

Fields

Modifier and Type	Field and Description
protected long	batchCounter
protected boolean	isEmptyKey Reduce sink key vector expressions.
protected boolean	isEmptyValue
protected BinarySortableSerializeWrite	keyBinarySortableSerializeWrite
protected HiveKey	keyWritable
protected TopNHash	reducerHash
protected int[]	reduceSinkKeyColumnMap
protected VectorExpression[]	reduceSinkKeyExpressions
protected TypeInfo[]	reduceSinkKeyTypeInfos
protected int[]	reduceSinkValueColumnMap
protected VectorExpression[]	reduceSinkValueExpressions
protected TypeInfo[]	reduceSinkValueTypeInfos
protected boolean	reduceSkipTag
protected byte	reduceTagByte
protected String	taskName
protected org.apache.hadoop.io.BytesWritable	valueBytesWritable
protected LazyBinarySerializeWrite	valueLazyBinarySerializeWrite
protected ByteStream.Output	valueOutput
protected VectorSerializeRow<LazyBinarySerializeWrite>	valueVectorSerializeRow
protected VectorizationContext	vContext
protected VectorReduceSinkDesc	vectorDesc
protected VectorReduceSinkInfo	vectorReduceSinkInfo Information about our native vectorized reduce sink created by the Vectorizer class during it decision process and useful for execution.

Fields inherited from class org.apache.hadoop.hive.ql.exec.Operator

abortOp, alias, asyncInitOperations, cContext, childOperators, childOperatorsArray, childOperatorsTag, conf, CONTEXT_NAME_KEY, done, groupKeyObject, HIVE_COUNTER_CREATED_DYNAMIC_PARTITIONS, HIVE_COUNTER_CREATED_FILES, HIVE_COUNTER_FATAL, id, inputObjInspectors, numRows, operatorId, outputObjInspector, parentOperators, reporter, runTimeNumRows, state, statsMap

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	VectorReduceSinkCommonOperator() Kryo ctor.
	VectorReduceSinkCommonOperator(CompilationOpContext ctx)
	VectorReduceSinkCommonOperator(CompilationOpContext ctx, OperatorDesc conf, VectorizationContext vContext, VectorDesc vectorDesc)

Method Summary

Modifier and Type	Method and Description
protected void	closeOp(boolean abort) Operator specific close routine.
void	collect(byte[] key, byte[] value, int hash)
protected void	collect(HiveKey keyWritable, org.apache.hadoop.io.BytesWritable valueWritable)
VectorizationContext	getInputVectorizationContext()
boolean	getIsReduceSink()
String	getName()
static String	getOperatorName()
VectorizationContext	getOutputVectorizationContext()
String	getReduceOutputName()
OperatorType	getType() Return the type of the specific operator among the types in OperatorType.
VectorDesc	getVectorDesc()
protected void	initializeEmptyKey(int tag)
protected void	initializeOp(org.apache.hadoop.conf.Configuration hconf) Operator specific initialization.
void	setOutputCollector(org.apache.hadoop.mapred.OutputCollector _out)

Methods inherited from class org.apache.hadoop.hive.ql.exec.Operator

abort, acceptLimitPushdown, allInitializedParentsAreClosed, areAllParentsInitialized, augmentPlan, cleanUpInputFileChanged, cleanUpInputFileChangedOp, clone, cloneOp, cloneRecursiveChildren, close, columnNamesRowResolvedCanBeObtained, completeInitializationOp, createDummy, defaultEndGroup, defaultStartGroup, dump, dump, endGroup, flush, flushRecursive, forward, getAdditionalCounters, getChildOperators, getChildren, getColumnExprMap, getCompilationOpContext, getConf, getConfiguration, getCounterName, getDone, getExecContext, getGroupKeyObject, getIdentifier, getInputObjInspectors, getMarker, getNextCntr, getNumChild, getNumParent, getOperatorId, getOpTraits, getOutputObjInspector, getParentOperators, getSchema, getStatistics, getStats, initEvaluators, initEvaluators, initEvaluatorsAndReturnStruct, initialize, initialize, initializeChildren, initializeLocalWork, initOperatorId, isUseBucketizedHiveInputFormat, jobClose, jobCloseOp, logicalEquals, logicalEqualsTree, logStats, opAllowedAfterMapJoin, opAllowedBeforeMapJoin, opAllowedBeforeSortMergeJoin, opAllowedConvertMapJoin, passExecContext, preorderMap, process, processGroup, removeChild, removeChildAndAdoptItsChildren, removeParent, removeParents, replaceChild, replaceParent, replaceTabAlias, reset, setAlias, setChildOperators, setColumnExprMap, setCompilationOpContext, setConf, setDone, setExecContext, setGroupKeyObject, setInputContext, setInputObjInspectors, setMarker, setNextVectorBatchGroupStatus, setOpTraits, setParentOperators, setReporter, setSchema, setStatistics, setUseBucketizedHiveInputFormat, startGroup, supportAutomaticSortMergeJoin, supportSkewJoinOptimization, supportUnionRemoveOptimization, toString, toString, toString, vectorForward

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

vectorReduceSinkInfo

protected VectorReduceSinkInfo vectorReduceSinkInfo

Information about our native vectorized reduce sink created by the Vectorizer class during it decision process and useful for execution.

vContext

protected VectorizationContext vContext

vectorDesc

protected VectorReduceSinkDesc vectorDesc

isEmptyKey

protected boolean isEmptyKey

Reduce sink key vector expressions.

reduceSinkKeyColumnMap

protected int[] reduceSinkKeyColumnMap

reduceSinkKeyTypeInfos

protected TypeInfo[] reduceSinkKeyTypeInfos

reduceSinkKeyExpressions

protected VectorExpression[] reduceSinkKeyExpressions

isEmptyValue

protected boolean isEmptyValue

reduceSinkValueColumnMap

protected int[] reduceSinkValueColumnMap

reduceSinkValueTypeInfos

protected TypeInfo[] reduceSinkValueTypeInfos

reduceSinkValueExpressions

protected VectorExpression[] reduceSinkValueExpressions

reduceSkipTag

protected transient boolean reduceSkipTag

reduceTagByte

protected transient byte reduceTagByte

keyBinarySortableSerializeWrite

protected transient BinarySortableSerializeWrite keyBinarySortableSerializeWrite

valueLazyBinarySerializeWrite

protected transient LazyBinarySerializeWrite valueLazyBinarySerializeWrite

valueVectorSerializeRow

protected transient VectorSerializeRow<LazyBinarySerializeWrite> valueVectorSerializeRow

valueOutput

protected transient ByteStream.Output valueOutput

keyWritable

protected transient HiveKey keyWritable

valueBytesWritable

protected transient org.apache.hadoop.io.BytesWritable valueBytesWritable

reducerHash

protected transient TopNHash reducerHash

taskName

protected transient String taskName

batchCounter

protected transient long batchCounter

Constructor Detail

VectorReduceSinkCommonOperator

protected VectorReduceSinkCommonOperator()

Kryo ctor.

VectorReduceSinkCommonOperator

public VectorReduceSinkCommonOperator(CompilationOpContext ctx)

VectorReduceSinkCommonOperator

public VectorReduceSinkCommonOperator(CompilationOpContext ctx,
                                      OperatorDesc conf,
                                      VectorizationContext vContext,
                                      VectorDesc vectorDesc)
                               throws HiveException

Throws:

HiveException

Method Detail

initializeOp

protected void initializeOp(org.apache.hadoop.conf.Configuration hconf)
                     throws HiveException

Description copied from class: Operator

Operator specific initialization.

Overrides:

initializeOp in class Operator<ReduceSinkDesc>

Throws:

HiveException

initializeEmptyKey

protected void initializeEmptyKey(int tag)

collect

public void collect(byte[] key,
                    byte[] value,
                    int hash)
             throws IOException

Specified by:

collect in interface TopNHash.BinaryCollector

Throws:

IOException

collect

protected void collect(HiveKey keyWritable,
                       org.apache.hadoop.io.BytesWritable valueWritable)
                throws HiveException,
                       IOException

Throws:

HiveException

IOException

closeOp

protected void closeOp(boolean abort)
                throws HiveException

Description copied from class: Operator

Operator specific close routine. Operators which inherents this class should overwrite this funtion for their specific cleanup routine.

Overrides:

closeOp in class Operator<ReduceSinkDesc>

Throws:

HiveException

getName

public String getName()

Specified by:

getName in interface Node

Overrides:

getName in class TerminalOperator<ReduceSinkDesc>

Returns:

the name of the operator

getOperatorName

public static String getOperatorName()

getType

public OperatorType getType()

Description copied from class: Operator

Return the type of the specific operator among the types in OperatorType.

Specified by:

getType in class Operator<ReduceSinkDesc>

Returns:

OperatorType.*

getOutputVectorizationContext

public VectorizationContext getOutputVectorizationContext()

Specified by:

getOutputVectorizationContext in interface VectorizationContextRegion

getIsReduceSink

public boolean getIsReduceSink()

Overrides:

getIsReduceSink in class Operator<ReduceSinkDesc>

getReduceOutputName

public String getReduceOutputName()

Overrides:

getReduceOutputName in class Operator<ReduceSinkDesc>

setOutputCollector

public void setOutputCollector(org.apache.hadoop.mapred.OutputCollector _out)

Overrides:

setOutputCollector in class Operator<ReduceSinkDesc>

getInputVectorizationContext

public VectorizationContext getInputVectorizationContext()

Specified by:

getInputVectorizationContext in interface VectorizationOperator

getVectorDesc

public VectorDesc getVectorDesc()

Specified by:

getVectorDesc in interface VectorizationOperator