org.apache.hadoop.hive.ql.optimizer

Class ParallelEdgeFixer

java.lang.Object

org.apache.hadoop.hive.ql.optimizer.Transform

org.apache.hadoop.hive.ql.optimizer.ParallelEdgeFixer

public class ParallelEdgeFixer
extends Transform

Inserts an extra RS to avoid parallel edges. For mapjoins, semijoins its less costly to sometimes reshuffle the existing data - than computing it from scratch. Parallel edges are introduced by the SharedWorkOptimizer in case this fixer could patch them up. +--------------------+ +--------------------+ | | | | | [...] [...] | | [...] [...] | | | | | | | | | | +-----+ +-----+ | | +-----+ +-----+ | | |RS_1 | |RS_2 | | | |RS_1 | |RS_2 | | | +-----+ +-----+ | | +-----+ +-----+ | | | | | | | | | +---- | ------ | ----+ +---- | ------ | ----+ | | | | | | | +--- | ---+ | | | | +-----+ | | | $gt; | | |RS_T | | | | | | +-----+ | | | | +--- | ---+ | | | | +---- | ------ | ----+ +---- | ------ | ----+ | +-----+ +-----+ | | +-----+ +-----+ | | |OP_1 | |OP_2 | | | |OP_1 | |OP_2 | | | +-----+ +-----+ | | +-----+ +-----+ | | | | | | | | | | [...] [...] | | [...] [...] | | | | | +--------------------+ +--------------------+

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor and Description
ParallelEdgeFixer()

Method Summary

Modifier and Type	Method and Description
static Optional<Set<String>>	colMappingInverseKeys(ReduceSinkOperator rs)
boolean	isParallelEdgeSupported(org.apache.calcite.util.Pair<Operator<?>,Operator<?>> pair)
ParseContext	transform(ParseContext pctx) All transformation steps implement this interface.

Methods inherited from class org.apache.hadoop.hive.ql.optimizer.Transform

beginPerfLogging, endPerfLogging, endPerfLogging

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected static final org.slf4j.Logger LOG

Constructor Detail

ParallelEdgeFixer

public ParallelEdgeFixer()

Method Detail

transform

public ParseContext transform(ParseContext pctx)
                       throws SemanticException

Description copied from class: Transform

All transformation steps implement this interface.

Specified by:

transform in class Transform

Parameters:

pctx - input parse context

Returns:

ParseContext

Throws:

SemanticException

isParallelEdgeSupported

public boolean isParallelEdgeSupported(org.apache.calcite.util.Pair<Operator<?>,Operator<?>> pair)

colMappingInverseKeys

public static Optional<Set<String>> colMappingInverseKeys(ReduceSinkOperator rs)