org.apache.tez.dag.app

Class TaskCommunicatorContextImpl

java.lang.Object

org.apache.tez.dag.app.TaskCommunicatorContextImpl

All Implemented Interfaces:

VertexStateUpdateListener, org.apache.tez.serviceplugins.api.ServicePluginContextBase, TaskCommunicatorContext

@InterfaceAudience.Private
public class TaskCommunicatorContextImpl
extends Object
implements TaskCommunicatorContext, VertexStateUpdateListener

Constructor Summary

Constructors

Constructor and Description
TaskCommunicatorContextImpl(AppContext appContext, TaskCommunicatorManager taskCommunicatorManager, org.apache.tez.dag.api.UserPayload userPayload, int taskCommunicatorIndex)

Method Summary

Modifier and Type	Method and Description
boolean	canCommit(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId) Check whether a running attempt can commit.
void	containerAlive(org.apache.hadoop.yarn.api.records.ContainerId containerId) Inform the framework that a container is alive.
void	dagCompleteEnd()
void	dagCompleteStart(DAG dag)
org.apache.hadoop.security.Credentials	getAMCredentials() Get credentials associated with the AppMaster
org.apache.hadoop.yarn.api.records.ApplicationAttemptId	getApplicationAttemptId() Get the application attempt id for the running application.
String	getCurrentAppIdentifier() Get an identifier for the executing context of the DAG.
org.apache.tez.serviceplugins.api.DagInfo	getCurrentDagInfo()
long	getDagStartTime() Get the start time for the currently executing DAG
long	getFirstAttemptStartTime(String vertexName, int taskIndex) Get the start time for the first attempt of the specified task
org.apache.tez.dag.api.UserPayload	getInitialUserPayload()
Iterable<String>	getInputVertexNames(String vertexName) Get the name of the Input vertices for the specified vertex.
int	getVertexCompletedTaskCount(String vertexName) Get the number of completed tasks for a given vertex
int	getVertexRunningTaskCount(String vertexName) Get the number of running tasks for a given vertex
int	getVertexTotalTaskCount(String vertexName) Get the total number of tasks in the given vertex
TaskHeartbeatResponse	heartbeat(TaskHeartbeatRequest request) Mechanism for a TaskCommunicator to provide updates on a running task, as well as receive new information which may need to be propagated to the task.
boolean	isKnownContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Check whether the container is known by the framework.
void	onStateUpdated(org.apache.tez.dag.api.event.VertexStateUpdate event)
void	registerForVertexStateUpdates(String vertexName, Set<org.apache.tez.dag.api.event.VertexState> stateSet) Register to get notifications on updates to the specified vertex.
void	reportError(org.apache.tez.serviceplugins.api.ServicePluginError servicePluginError, String message, org.apache.tez.serviceplugins.api.DagInfo dagInfo)
void	taskAlive(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId) Inform the framework that a task is alive.
void	taskFailed(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId, org.apache.tez.runtime.api.TaskFailureType taskFailureType, org.apache.tez.serviceplugins.api.TaskAttemptEndReason taskAttemptEndReason, String diagnostics) Inform the framework that a task has failed.
void	taskKilled(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId, org.apache.tez.serviceplugins.api.TaskAttemptEndReason taskAttemptEndReason, String diagnostics) Inform the framework that a task has been killed
void	taskStartedRemotely(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId) Inform the framework that the task has started execution
void	taskStartedRemotely(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId, org.apache.hadoop.yarn.api.records.ContainerId containerId) Inform the framework that the task has started execution Use TaskCommunicatorContext.taskSubmitted(TezTaskAttemptID, ContainerId) and TaskCommunicatorContext.taskStartedRemotely(TezTaskAttemptID) instead
void	taskSubmitted(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId, org.apache.hadoop.yarn.api.records.ContainerId containerId) Inform the framework that the task has been submitted for execution.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TaskCommunicatorContextImpl

public TaskCommunicatorContextImpl(AppContext appContext,
                                   TaskCommunicatorManager taskCommunicatorManager,
                                   org.apache.tez.dag.api.UserPayload userPayload,
                                   int taskCommunicatorIndex)

Method Detail

getInitialUserPayload

public org.apache.tez.dag.api.UserPayload getInitialUserPayload()

Specified by:

getInitialUserPayload in interface org.apache.tez.serviceplugins.api.ServicePluginContextBase

getApplicationAttemptId

public org.apache.hadoop.yarn.api.records.ApplicationAttemptId getApplicationAttemptId()

Description copied from interface: TaskCommunicatorContext

Get the application attempt id for the running application. Relevant when running under YARN

Specified by:

getApplicationAttemptId in interface TaskCommunicatorContext

Returns:

the applicationAttemptId for the running app

getAMCredentials

public org.apache.hadoop.security.Credentials getAMCredentials()

Description copied from interface: TaskCommunicatorContext

Get credentials associated with the AppMaster

Specified by:

getAMCredentials in interface TaskCommunicatorContext

Returns:

credentials

canCommit

public boolean canCommit(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId)
                  throws IOException

Description copied from interface: TaskCommunicatorContext

Check whether a running attempt can commit. This provides a leader election mechanism amongst multiple running attempts

Specified by:

canCommit in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the associated task attempt id

Returns:

whether the attempt can commit or not

Throws:

IOException

heartbeat

public TaskHeartbeatResponse heartbeat(TaskHeartbeatRequest request)
                                throws IOException,
                                       org.apache.tez.dag.api.TezException

Description copied from interface: TaskCommunicatorContext

Mechanism for a TaskCommunicator to provide updates on a running task, as well as receive new information which may need to be propagated to the task. This includes events generated by the task and events which need to be sent to the task This method must be invoked periodically to receive updates for a running task

Specified by:

heartbeat in interface TaskCommunicatorContext

Parameters:

request - the update from the running task.

Returns:

the response that is requried by the task.

Throws:

IOException

org.apache.tez.dag.api.TezException

isKnownContainer

public boolean isKnownContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: TaskCommunicatorContext

Check whether the container is known by the framework. The state of this container is irrelevant

Specified by:

isKnownContainer in interface TaskCommunicatorContext

Parameters:

containerId - the relevant container id

Returns:

true if the container is known, false if it isn't

taskAlive

public void taskAlive(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId)

Description copied from interface: TaskCommunicatorContext

Inform the framework that a task is alive. This needs to be invoked periodically to avoid the task attempt timing out. Invocations to heartbeat provides the same keep-alive functionality

Specified by:

taskAlive in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt

containerAlive

public void containerAlive(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: TaskCommunicatorContext

Inform the framework that a container is alive. This need to be invoked periodically to avoid the container attempt timing out. Invocations to heartbeat provides the same keep-alive functionality

Specified by:

containerAlive in interface TaskCommunicatorContext

Parameters:

containerId - the relevant container id

taskSubmitted

public void taskSubmitted(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId,
                          org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: TaskCommunicatorContext

Inform the framework that the task has been submitted for execution. The expectation is that the implementing TaskCommunicator will inform the framework about task submission, followed by the task starting.

Specified by:

taskSubmitted in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt id

containerId - the containerId in which the task attempt is running

taskStartedRemotely

public void taskStartedRemotely(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId)

Description copied from interface: TaskCommunicatorContext

Inform the framework that the task has started execution

Specified by:

taskStartedRemotely in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt id

taskStartedRemotely

public void taskStartedRemotely(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId,
                                org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: TaskCommunicatorContext

Inform the framework that the task has started execution Use TaskCommunicatorContext.taskSubmitted(TezTaskAttemptID, ContainerId) and TaskCommunicatorContext.taskStartedRemotely(TezTaskAttemptID) instead

Specified by:

taskStartedRemotely in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt id

taskKilled

public void taskKilled(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId,
                       org.apache.tez.serviceplugins.api.TaskAttemptEndReason taskAttemptEndReason,
                       @Nullable
                       String diagnostics)

Description copied from interface: TaskCommunicatorContext

Inform the framework that a task has been killed

Specified by:

taskKilled in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt id

taskAttemptEndReason - the reason for the task attempt being killed

diagnostics - any diagnostics messages which are relevant to the task attempt kill

taskFailed

public void taskFailed(org.apache.tez.dag.records.TezTaskAttemptID taskAttemptId,
                       org.apache.tez.runtime.api.TaskFailureType taskFailureType,
                       org.apache.tez.serviceplugins.api.TaskAttemptEndReason taskAttemptEndReason,
                       @Nullable
                       String diagnostics)

Description copied from interface: TaskCommunicatorContext

Inform the framework that a task has failed. This, at the moment, is always treated as a an error which will cause a retry of the task to be triggered, if there are enough retry attempts left.

Specified by:

taskFailed in interface TaskCommunicatorContext

Parameters:

taskAttemptId - the relevant task attempt id

taskFailureType - the type of the error

taskAttemptEndReason - the reason for the task failure

diagnostics - any diagnostics messages which are relevant to the task attempt failure

registerForVertexStateUpdates

public void registerForVertexStateUpdates(String vertexName,
                                          @Nullable
                                          Set<org.apache.tez.dag.api.event.VertexState> stateSet)

Description copied from interface: TaskCommunicatorContext

Register to get notifications on updates to the specified vertex. Notifications will be sent via InputInitializer.onVertexStateUpdated(org.apache.tez.dag.api.event.VertexStateUpdate)

This method can only be invoked once. Duplicate invocations will result in an error.

Specified by:

registerForVertexStateUpdates in interface TaskCommunicatorContext

Parameters:

vertexName - the vertex name for which notifications are required.

stateSet - the set of states for which notifications are required. null implies all

getCurrentAppIdentifier

public String getCurrentAppIdentifier()

Description copied from interface: TaskCommunicatorContext

Get an identifier for the executing context of the DAG.

Specified by:

getCurrentAppIdentifier in interface TaskCommunicatorContext

Returns:

a String identifier for the exeucting context.

getCurrentDagInfo

@Nullable
public org.apache.tez.serviceplugins.api.DagInfo getCurrentDagInfo()

Specified by:

getCurrentDagInfo in interface org.apache.tez.serviceplugins.api.ServicePluginContextBase

getInputVertexNames

public Iterable<String> getInputVertexNames(String vertexName)

Description copied from interface: TaskCommunicatorContext

Get the name of the Input vertices for the specified vertex. Root Inputs are not returned.

Specified by:

getInputVertexNames in interface TaskCommunicatorContext

Parameters:

vertexName - the vertex for which source vertex names will be returned

Returns:

an Iterable containing the list of input vertices for the specified vertex

getVertexTotalTaskCount

public int getVertexTotalTaskCount(String vertexName)

Description copied from interface: TaskCommunicatorContext

Get the total number of tasks in the given vertex

Specified by:

getVertexTotalTaskCount in interface TaskCommunicatorContext

Parameters:

vertexName - the relevant vertex name

Returns:

total number of tasks in this vertex

getVertexCompletedTaskCount

public int getVertexCompletedTaskCount(String vertexName)

Description copied from interface: TaskCommunicatorContext

Get the number of completed tasks for a given vertex

Specified by:

getVertexCompletedTaskCount in interface TaskCommunicatorContext

Parameters:

vertexName - the vertex name

Returns:

the number of completed tasks for the vertex

getVertexRunningTaskCount

public int getVertexRunningTaskCount(String vertexName)

Description copied from interface: TaskCommunicatorContext

Get the number of running tasks for a given vertex

Specified by:

getVertexRunningTaskCount in interface TaskCommunicatorContext

Parameters:

vertexName - the vertex name

Returns:

the number of running tasks for the vertex

getFirstAttemptStartTime

public long getFirstAttemptStartTime(String vertexName,
                                     int taskIndex)

Description copied from interface: TaskCommunicatorContext

Get the start time for the first attempt of the specified task

Specified by:

getFirstAttemptStartTime in interface TaskCommunicatorContext

Parameters:

vertexName - the vertex to which the task belongs

taskIndex - the index of the task

Returns:

the start time for the first attempt of the task

getDagStartTime

public long getDagStartTime()

Description copied from interface: TaskCommunicatorContext

Get the start time for the currently executing DAG

Specified by:

getDagStartTime in interface TaskCommunicatorContext

Returns:

time when the current dag started executing

reportError

public void reportError(@Nonnull
                        org.apache.tez.serviceplugins.api.ServicePluginError servicePluginError,
                        String message,
                        org.apache.tez.serviceplugins.api.DagInfo dagInfo)

Specified by:

reportError in interface org.apache.tez.serviceplugins.api.ServicePluginContextBase

onStateUpdated

public void onStateUpdated(org.apache.tez.dag.api.event.VertexStateUpdate event)

Specified by:

onStateUpdated in interface VertexStateUpdateListener

dagCompleteStart

@InterfaceAudience.Private
public void dagCompleteStart(DAG dag)

dagCompleteEnd

public void dagCompleteEnd()