org.apache.hadoop.hive.ql.parse

Class TaskCompiler

java.lang.Object

org.apache.hadoop.hive.ql.parse.TaskCompiler

Direct Known Subclasses:

MapReduceCompiler, TezCompiler

public abstract class TaskCompiler
extends Object

TaskCompiler is a the base class for classes that compile operator pipelines into tasks.

Field Summary

Fields

Modifier and Type	Field and Description
protected HiveConf	conf
protected SessionState.LogHelper	console
protected Hive	db
protected org.slf4j.Logger	LOG
protected QueryState	queryState

Constructor Summary

Constructors

Constructor and Description
TaskCompiler()

Method Summary

Modifier and Type	Method and Description
void	compile(ParseContext pCtx, List<Task<?>> rootTasks, Set<ReadEntity> inputs, Set<WriteEntity> outputs)
protected abstract void	decideExecMode(List<Task<?>> rootTasks, Context ctx, GlobalLimitCtx globalLimitCtx)
protected void	genColumnStatsTask(BaseSemanticAnalyzer.AnalyzeRewriteContext analyzeRewrite, List<LoadFileDesc> loadFileWork, Map<String,StatsTask> map, int outerQueryLimit, int numBitVector) A helper function to generate a column stats task on top of map-red task.
protected abstract void	generateTaskTree(List<Task<?>> rootTasks, ParseContext pCtx, List<Task<MoveWork>> mvTask, Set<ReadEntity> inputs, Set<WriteEntity> outputs)
ParseContext	getParseContext(ParseContext pCtx, List<Task<?>> rootTasks) Create a clone of the parse context
String	getTableOrMVSuffix(ParseContext pCtx, boolean createTableOrMVUseSuffix)
void	init(QueryState queryState, SessionState.LogHelper console, Hive db)
protected void	optimizeOperatorPlan(ParseContext pCtxSet)
protected abstract void	optimizeTaskPlan(List<Task<?>> rootTasks, ParseContext pCtx, Context ctx)
protected void	runDynPartitionSortOptimizations(ParseContext parseContext, HiveConf hConf)
protected abstract void	setInputFormat(Task<?> rootTask)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected final org.slf4j.Logger LOG

db

protected Hive db

console

protected SessionState.LogHelper console

queryState

protected QueryState queryState

conf

protected HiveConf conf

Constructor Detail

TaskCompiler

public TaskCompiler()

Method Detail

init

public void init(QueryState queryState,
                 SessionState.LogHelper console,
                 Hive db)

compile

public void compile(ParseContext pCtx,
                    List<Task<?>> rootTasks,
                    Set<ReadEntity> inputs,
                    Set<WriteEntity> outputs)
             throws SemanticException

Throws:

SemanticException

getTableOrMVSuffix

public String getTableOrMVSuffix(ParseContext pCtx,
                                 boolean createTableOrMVUseSuffix)

genColumnStatsTask

protected void genColumnStatsTask(BaseSemanticAnalyzer.AnalyzeRewriteContext analyzeRewrite,
                                  List<LoadFileDesc> loadFileWork,
                                  Map<String,StatsTask> map,
                                  int outerQueryLimit,
                                  int numBitVector)
                           throws SemanticException

A helper function to generate a column stats task on top of map-red task. The column stats task fetches from the output of the map-red task, constructs the column stats object and persists it to the metastore. This method generates a plan with a column stats task on top of map-red task and sets up the appropriate metadata to be used during execution.

Throws:

SemanticException

decideExecMode

protected abstract void decideExecMode(List<Task<?>> rootTasks,
                                       Context ctx,
                                       GlobalLimitCtx globalLimitCtx)
                                throws SemanticException

Throws:

SemanticException

optimizeOperatorPlan

protected void optimizeOperatorPlan(ParseContext pCtxSet)
                             throws SemanticException

Throws:

SemanticException

optimizeTaskPlan

protected abstract void optimizeTaskPlan(List<Task<?>> rootTasks,
                                         ParseContext pCtx,
                                         Context ctx)
                                  throws SemanticException

Throws:

SemanticException

setInputFormat

protected abstract void setInputFormat(Task<?> rootTask)

generateTaskTree

protected abstract void generateTaskTree(List<Task<?>> rootTasks,
                                         ParseContext pCtx,
                                         List<Task<MoveWork>> mvTask,
                                         Set<ReadEntity> inputs,
                                         Set<WriteEntity> outputs)
                                  throws SemanticException

Throws:

SemanticException

runDynPartitionSortOptimizations

protected void runDynPartitionSortOptimizations(ParseContext parseContext,
                                                HiveConf hConf)
                                         throws SemanticException

Throws:

SemanticException

getParseContext

public ParseContext getParseContext(ParseContext pCtx,
                                    List<Task<?>> rootTasks)

Create a clone of the parse context