Task (Hive Query Language 4.0.0-alpha-2 API)

java.lang.Object
- org.apache.hadoop.hive.ql.exec.Task<T>

All Implemented Interfaces:

Serializable, Node

Direct Known Subclasses:

AckTask, AtlasDumpTask, AtlasLoadTask, ColumnStatsUpdateTask, ColumnTruncateTask, ConditionalTask, CopyTask, DDLTask, DependencyCollectionTask, DirCopyTask, ExecDriver, ExplainSQRewriteTask, ExplainTask, ExportTask, FetchTask, MapredLocalTask, MergeFileTask, MoveTask, RangerDenyTask, RangerDumpTask, RangerLoadTask, ReplCopyTask, ReplDumpTask, ReplLoadTask, ReplStateLogTask, ReplTxnTask, ScheduledQueryMaintenanceTask, StatsTask, TezTask
```
public abstract class Task<T extends Serializable>
extends Object
implements Serializable, Node
```
Task implementation.

See Also:

Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`static class`	`Task.FeedType`
`static class`	`Task.TaskState` Order of the States here is important as the ordinal values are used determine the progression of taskState over its lifeCycle which is then used to make some decisions in Driver.execute

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`BACKUP_COMMON_JOIN`
`protected List<Task<?>>`	`backupChildrenTasks`
`protected Task<?>`	`backupTask`
`protected List<Task<?>>`	`childTasks`
`protected boolean`	`clonedConf`
`static int`	`COMMON_JOIN`
`protected HiveConf`	`conf`
`protected SessionState.LogHelper`	`console`
`protected Context`	`context`
`static int`	`CONVERTED_MAPJOIN`
`static int`	`CONVERTED_MAPJOIN_LOCAL`
`static int`	`CONVERTED_SORTMERGEJOIN`
`protected DeferredWorkContext`	`deferredWorkContext`
`protected List<Task<?>>`	`feedSubscribers`
`static int`	`HINTED_MAPJOIN`
`static int`	`HINTED_MAPJOIN_LOCAL`
`protected String`	`id`
`protected String`	`jobID`
`protected static org.slf4j.Logger`	`LOG`
`static int`	`MAPJOIN_ONLY_NOBACKUP`
`static int`	`NO_TAG`
`protected List<Task<?>>`	`parentTasks`
`QueryDisplay`	`queryDisplay`
`protected QueryPlan`	`queryPlan`
`protected QueryState`	`queryState`
`protected boolean`	`rootTask`
`HashMap<String,Long>`	`taskCounters`
`TaskHandle`	`taskHandle`
`protected TaskQueue`	`taskQueue`
`protected int`	`taskTag`
`protected T`	`work`

Constructor Summary

Constructors
Constructor and Description

Task()

Constructors
Constructor and Description
`Task()`

Method Summary

All Methods Static Methods Instance Methods Abstract Methods Concrete Methods
Modifier and Type	Method and Description
`boolean`	`addDependentTask(Task<?> dependent)` Add a dependent task on the current task.
`boolean`	`canExecuteInParallel()`
`protected void`	`cloneConf()`
`boolean`	`done()`
`boolean`	`equals(Object obj)`
`abstract int`	`execute()` This method is overridden in each Task.
`int`	`executeTask(HiveHistory hiveHistory)` This method is called in the Driver on every task.
`static List<Task<?>>`	`findLeafs(List<Task<?>> rootTasks)`
`Task<?>`	`getAndInitBackupTask()`
`List<Task<?>>`	`getBackupChildrenTasks()`
`Task<?>`	`getBackupTask()`
`List<? extends Node>`	`getChildren()`
`List<Task<?>>`	`getChildTasks()`
`HiveConf`	`getConf()`
`Context`	`getContext()`
`HashMap<String,Long>`	`getCounters()`
`DeferredWorkContext`	`getDeferredWorkContext()`
`List<Task<?>>`	`getDependentTasks()` The default dependent tasks are just child tasks, but different types could implement their own (e.g.
`String`	`getDiagnosticsMessage()`
`Throwable`	`getException()`
`String`	`getExternalHandle()`
`List<Task<?>>`	`getFeedSubscribers()`
`protected Hive`	`getHive()`
`String`	`getId()`
`boolean`	`getInitialized()`
`String`	`getJobID()`
`Collection<MapWork>`	`getMapWork()`
`int`	`getNumChild()`
`int`	`getNumParent()`
`List<Task<?>>`	`getParentTasks()`
`QueryPlan`	`getQueryPlan()`
`QueryState`	`getQueryState()`
`boolean`	`getQueued()`
`Operator<? extends OperatorDesc>`	`getReducer(MapWork work)`
`String`	`getStatusMessage()`
`TaskHandle`	`getTaskHandle()`
`TaskQueue`	`getTaskQueue()`
`Task.TaskState`	`getTaskState()`
`int`	`getTaskTag()`
`Collection<Operator<? extends OperatorDesc>>`	`getTopOperators()`
`HiveTxnManager`	`getTxnMgr()`
`abstract StageType`	`getType()` Should be overridden to return the type of the specific task among the types in StageType.
`T`	`getWork()`
`int`	`hashCode()`
`boolean`	`hasReduce()`
`void`	`initialize(QueryState queryState, QueryPlan queryPlan, TaskQueue taskQueue, Context context)`
`boolean`	`isFetchSource()`
`boolean`	`isLocalMode()`
`boolean`	`isMapRedLocalTask()`
`boolean`	`isMapRedTask()`
`boolean`	`isNotInitialized()`
`boolean`	`isRootTask()`
`boolean`	`isRunnable()`
`protected void`	`pushFeed(Task.FeedType feedType, Object feedValue)`
`protected void`	`receiveFeed(Task.FeedType feedType, Object feedValue)`
`void`	`removeDependentTask(Task<?> dependent)` Remove the dependent task.
`void`	`removeFromChildrenTasks()`
`boolean`	`requireLock()`
`void`	`setBackupChildrenTasks(List<Task<?>> backupChildrenTasks)`
`void`	`setBackupTask(Task<?> backupTask)`
`void`	`setChildTasks(List<Task<?>> childTasks)`
`void`	`setConf(HiveConf conf)`
`void`	`setConsole(SessionState.LogHelper console)`
`void`	`setDeferredWorkContext(DeferredWorkContext deferredWorkContext)`
`void`	`setDiagnosticMessage(String diagnosticMesg)`
`void`	`setDone()`
`void`	`setException(Throwable ex)`
`void`	`setFeedSubscribers(List<Task<?>> s)`
`void`	`setFetchSource(boolean fetchSource)`
`void`	`setId(String id)`
`void`	`setInitialized()`
`void`	`setLocalMode(boolean isLocalMode)`
`void`	`setParentTasks(List<Task<?>> parentTasks)`
`void`	`setQueryDisplay(QueryDisplay queryDisplay)`
`void`	`setQueryPlan(QueryPlan queryPlan)`
`void`	`setQueued()`
`void`	`setRootTask(boolean rootTask)`
`void`	`setStarted()`
`protected void`	`setState(Task.TaskState state)`
`void`	`setStatusMessage(String statusMessage)`
`void`	`setTaskQueue(TaskQueue taskQueue)`
`void`	`setTaskTag(int taskTag)`
`void`	`setWork(T work)`
`void`	`shutdown()`
`boolean`	`started()`
`void`	`subscribeFeed(Task<?> publisher)` Subscribe the feed of publisher.
`String`	`toString()`
`protected void`	`updateStatusInQueryDisplay()`
`void`	`updateTaskMetrics(org.apache.hadoop.hive.common.metrics.common.Metrics metrics)` Provide metrics on the type and number of tasks executed by the HiveServer

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.hive.ql.lib.Node
getName

Field Detail

taskCounters

public transient HashMap<String,Long> taskCounters

taskHandle
```
public transient TaskHandle taskHandle
```

conf
```
protected transient HiveConf conf
```

queryState

protected transient QueryState queryState

console

protected transient SessionState.LogHelper console

queryPlan

protected transient QueryPlan queryPlan

taskQueue

protected transient TaskQueue taskQueue

context
```
protected transient Context context
```

clonedConf
```
protected transient boolean clonedConf
```

jobID
```
protected transient String jobID
```

backupTask
```
protected Task<?> backupTask
```

backupChildrenTasks

protected List<Task<?>> backupChildrenTasks

LOG

protected static transient org.slf4j.Logger LOG

taskTag
```
protected int taskTag
```

deferredWorkContext

protected DeferredWorkContext deferredWorkContext

NO_TAG
```
public static final int NO_TAG
```
See Also:

Constant Field Values

COMMON_JOIN
```
public static final int COMMON_JOIN
```
See Also:

Constant Field Values

HINTED_MAPJOIN
```
public static final int HINTED_MAPJOIN
```
See Also:

Constant Field Values

HINTED_MAPJOIN_LOCAL
```
public static final int HINTED_MAPJOIN_LOCAL
```
See Also:

Constant Field Values

CONVERTED_MAPJOIN
```
public static final int CONVERTED_MAPJOIN
```
See Also:

Constant Field Values

CONVERTED_MAPJOIN_LOCAL
```
public static final int CONVERTED_MAPJOIN_LOCAL
```
See Also:

Constant Field Values

BACKUP_COMMON_JOIN
```
public static final int BACKUP_COMMON_JOIN
```
See Also:

Constant Field Values

MAPJOIN_ONLY_NOBACKUP
```
public static final int MAPJOIN_ONLY_NOBACKUP
```
See Also:

Constant Field Values

CONVERTED_SORTMERGEJOIN
```
public static final int CONVERTED_SORTMERGEJOIN
```
See Also:

Constant Field Values

queryDisplay
```
public QueryDisplay queryDisplay
```

feedSubscribers

protected transient List<Task<?>> feedSubscribers

id
```
protected String id
```

work
```
protected T extends Serializable work
```

rootTask
```
protected boolean rootTask
```

childTasks
```
protected List<Task<?>> childTasks
```

parentTasks
```
protected List<Task<?>> parentTasks
```

Constructor Detail
- Task
```
public Task()
```

Method Detail

setDiagnosticMessage

public void setDiagnosticMessage(String diagnosticMesg)

getDiagnosticsMessage
```
public String getDiagnosticsMessage()
```

setStatusMessage

public void setStatusMessage(String statusMessage)

getStatusMessage
```
public String getStatusMessage()
```

getTaskHandle
```
public TaskHandle getTaskHandle()
```

initialize

public void initialize(QueryState queryState,
                       QueryPlan queryPlan,
                       TaskQueue taskQueue,
                       Context context)

setQueryDisplay

public void setQueryDisplay(QueryDisplay queryDisplay)

updateStatusInQueryDisplay

protected void updateStatusInQueryDisplay()

setState

protected void setState(Task.TaskState state)

getHive
```
protected Hive getHive()
```

executeTask
```
public int executeTask(HiveHistory hiveHistory)
```
This method is called in the Driver on every task. It updates counters and calls execute(), which is overridden in each task

Returns:

return value of execute()

execute
```
public abstract int execute()
```
This method is overridden in each Task. TODO execute should return a TaskHandle.

Returns:

status of executing the task

isRootTask
```
public boolean isRootTask()
```

setRootTask

public void setRootTask(boolean rootTask)

setChildTasks

public void setChildTasks(List<Task<?>> childTasks)

getChildren
```
public List<? extends Node> getChildren()
```
Specified by:

getChildren in interface Node

getChildTasks
```
public List<Task<?>> getChildTasks()
```

getNumChild
```
public int getNumChild()
```

setParentTasks

public void setParentTasks(List<Task<?>> parentTasks)

getParentTasks
```
public List<Task<?>> getParentTasks()
```

getNumParent
```
public int getNumParent()
```

getBackupTask
```
public Task<?> getBackupTask()
```

setBackupTask

public void setBackupTask(Task<?> backupTask)

getBackupChildrenTasks

public List<Task<?>> getBackupChildrenTasks()

setBackupChildrenTasks

public void setBackupChildrenTasks(List<Task<?>> backupChildrenTasks)

getAndInitBackupTask
```
public Task<?> getAndInitBackupTask()
```

removeFromChildrenTasks
```
public void removeFromChildrenTasks()
```

getDependentTasks
```
public List<Task<?>> getDependentTasks()
```
The default dependent tasks are just child tasks, but different types could implement their own (e.g. ConditionalTask will use the listTasks as dependents).

Returns:

a list of tasks that are dependent on this task.

addDependentTask
```
public boolean addDependentTask(Task<?> dependent)
```
Add a dependent task on the current task. Return if the dependency already existed or is this a new one

Returns:

true if the task got added false if it already existed

findLeafs

public static List<Task<?>> findLeafs(List<Task<?>> rootTasks)

removeDependentTask
```
public void removeDependentTask(Task<?> dependent)
```
Remove the dependent task.

Parameters:

dependent - the task to remove

setStarted
```
public void setStarted()
```

started
```
public boolean started()
```

done
```
public boolean done()
```

setDone
```
public void setDone()
```

setQueued
```
public void setQueued()
```

getQueued
```
public boolean getQueued()
```

setInitialized
```
public void setInitialized()
```

getInitialized
```
public boolean getInitialized()
```

isNotInitialized
```
public boolean isNotInitialized()
```

isRunnable
```
public boolean isRunnable()
```

setConf
```
public void setConf(HiveConf conf)
```

getConf
```
public HiveConf getConf()
```

setWork
```
public void setWork(T work)
```

getWork
```
public T getWork()
```

getMapWork

public Collection<MapWork> getMapWork()

setId
```
public void setId(String id)
```

getId
```
public String getId()
```

getExternalHandle
```
public String getExternalHandle()
```

getTaskState
```
public Task.TaskState getTaskState()
```

isMapRedTask
```
public boolean isMapRedTask()
```

isMapRedLocalTask
```
public boolean isMapRedLocalTask()
```

getTopOperators

public Collection<Operator<? extends OperatorDesc>> getTopOperators()

hasReduce
```
public boolean hasReduce()
```

getReducer

public Operator<? extends OperatorDesc> getReducer(MapWork work)

getCounters

public HashMap<String,Long> getCounters()

getType
```
public abstract StageType getType()
```
Should be overridden to return the type of the specific task among the types in StageType.

Returns:

StageType.* or null if not overridden

subscribeFeed
```
public void subscribeFeed(Task<?> publisher)
```
Subscribe the feed of publisher. To prevent cycles, a task can only subscribe to its ancestor. Feed is a generic form of execution-time feedback (type, value) pair from one task to another task. Examples include dynamic partitions (which are only available at execution time). The MoveTask may pass the list of dynamic partitions to the StatsTask since after the MoveTask the list of dynamic partitions are lost (MoveTask moves them to the table's destination directory which is mixed with old partitions).

Parameters:

publisher - this feed provider.

getFeedSubscribers

public List<Task<?>> getFeedSubscribers()

setFeedSubscribers

public void setFeedSubscribers(List<Task<?>> s)

pushFeed

protected void pushFeed(Task.FeedType feedType,
                        Object feedValue)

receiveFeed

protected void receiveFeed(Task.FeedType feedType,
                           Object feedValue)

cloneConf
```
protected void cloneConf()
```

updateTaskMetrics
```
public void updateTaskMetrics(org.apache.hadoop.hive.common.metrics.common.Metrics metrics)
```
Provide metrics on the type and number of tasks executed by the HiveServer

Parameters:

metrics -

getTaskTag
```
public int getTaskTag()
```

setTaskTag
```
public void setTaskTag(int taskTag)
```

isLocalMode
```
public boolean isLocalMode()
```

setLocalMode

public void setLocalMode(boolean isLocalMode)

requireLock
```
public boolean requireLock()
```

getQueryPlan
```
public QueryPlan getQueryPlan()
```

getTaskQueue
```
public TaskQueue getTaskQueue()
```

setTaskQueue

public void setTaskQueue(TaskQueue taskQueue)

getContext
```
public Context getContext()
```

setQueryPlan

public void setQueryPlan(QueryPlan queryPlan)

getJobID
```
public String getJobID()
```

shutdown
```
public void shutdown()
```

getException
```
public Throwable getException()
```

setException

public void setException(Throwable ex)

setConsole

public void setConsole(SessionState.LogHelper console)

isFetchSource
```
public boolean isFetchSource()
```

setFetchSource

public void setFetchSource(boolean fetchSource)

getDeferredWorkContext

public DeferredWorkContext getDeferredWorkContext()

setDeferredWorkContext

public void setDeferredWorkContext(DeferredWorkContext deferredWorkContext)

toString
```
public String toString()
```
Overrides:

toString in class Object

hashCode
```
public int hashCode()
```
Overrides:

hashCode in class Object

equals
```
public boolean equals(Object obj)
```
Overrides:

equals in class Object

canExecuteInParallel
```
public boolean canExecuteInParallel()
```

getQueryState
```
public QueryState getQueryState()
```

getTxnMgr
```
public HiveTxnManager getTxnMgr()
```

Class Task<T extends Serializable>

Nested Class Summary

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Methods inherited from interface org.apache.hadoop.hive.ql.lib.Node

Field Detail

taskCounters

taskHandle

conf

queryState

console

queryPlan

taskQueue

context

clonedConf

jobID

backupTask

backupChildrenTasks

LOG

taskTag

deferredWorkContext

NO_TAG

COMMON_JOIN

HINTED_MAPJOIN

HINTED_MAPJOIN_LOCAL

CONVERTED_MAPJOIN

CONVERTED_MAPJOIN_LOCAL

BACKUP_COMMON_JOIN

MAPJOIN_ONLY_NOBACKUP

CONVERTED_SORTMERGEJOIN

queryDisplay

feedSubscribers

id

work

rootTask

childTasks

parentTasks

Constructor Detail

Task

Method Detail

setDiagnosticMessage

getDiagnosticsMessage

setStatusMessage

getStatusMessage

getTaskHandle

initialize

setQueryDisplay

updateStatusInQueryDisplay

setState

getHive

executeTask

execute

isRootTask

setRootTask

setChildTasks

getChildren

getChildTasks

getNumChild

setParentTasks

getParentTasks

getNumParent

getBackupTask

setBackupTask

getBackupChildrenTasks

setBackupChildrenTasks

getAndInitBackupTask

removeFromChildrenTasks

getDependentTasks

addDependentTask

findLeafs

removeDependentTask

setStarted

started

done

setDone

setQueued

getQueued

setInitialized