org.apache.tez.dag.app.dag.speculation.legacy

Class LegacyTaskRuntimeEstimator

java.lang.Object

org.apache.tez.dag.app.dag.speculation.legacy.LegacyTaskRuntimeEstimator

All Implemented Interfaces:

TaskRuntimeEstimator

public class LegacyTaskRuntimeEstimator
extends Object

Runtime estimator that uses a simple scheme of estimating task attempt runtime based on current elapsed runtime and reported progress.

Field Summary

Fields

Modifier and Type	Field and Description
protected Set<Task>	doneTasks
protected Map<org.apache.tez.dag.records.TezTaskAttemptID,Long>	startTimes
protected DataStatistics	taskStatistics
protected Vertex	vertex

Constructor Summary

Constructors

Constructor and Description
LegacyTaskRuntimeEstimator()

Method Summary

Modifier and Type	Method and Description
long	attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)
void	contextualize(org.apache.hadoop.conf.Configuration conf, Vertex vertexP)
protected DataStatistics	dataStatisticsForTask(org.apache.tez.dag.records.TezTaskID taskID)
void	enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id, long timestamp)
long	estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID attemptID) Estimate a task attempt's total runtime.
long	newAttemptEstimatedRuntime() Estimates how long a new attempt on this task will take if we start one now
long	runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID attemptID) Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TezTaskAttemptID)
long	thresholdRuntime(org.apache.tez.dag.records.TezTaskID taskID) Find a maximum reasonable execution wallclock time.
void	updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID attemptID, TaskAttemptState state, long timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.tez.dag.app.dag.speculation.legacy.TaskRuntimeEstimator

hasStagnatedProgress

Field Detail

vertex

protected Vertex vertex

startTimes

protected final Map<org.apache.tez.dag.records.TezTaskAttemptID,Long> startTimes

taskStatistics

protected final DataStatistics taskStatistics

doneTasks

protected final Set<Task> doneTasks

Constructor Detail

LegacyTaskRuntimeEstimator

public LegacyTaskRuntimeEstimator()

Method Detail

updateAttempt

public void updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID attemptID,
                          TaskAttemptState state,
                          long timestamp)

Specified by:

updateAttempt in interface TaskRuntimeEstimator

estimatedRuntime

public long estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Description copied from interface: TaskRuntimeEstimator

Estimate a task attempt's total runtime. Includes the time already elapsed.

Parameters:

attemptID - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

runtimeEstimateVariance

public long runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Description copied from interface: TaskRuntimeEstimator

Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TezTaskAttemptID)

Parameters:

attemptID - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

enrollAttempt

public void enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id,
                          long timestamp)

Specified by:

enrollAttempt in interface TaskRuntimeEstimator

attemptEnrolledTime

public long attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

Specified by:

attemptEnrolledTime in interface TaskRuntimeEstimator

contextualize

public void contextualize(org.apache.hadoop.conf.Configuration conf,
                          Vertex vertexP)

Specified by:

contextualize in interface TaskRuntimeEstimator

dataStatisticsForTask

protected DataStatistics dataStatisticsForTask(org.apache.tez.dag.records.TezTaskID taskID)

thresholdRuntime

public long thresholdRuntime(org.apache.tez.dag.records.TezTaskID taskID)

Description copied from interface: TaskRuntimeEstimator

Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.

Specified by:

thresholdRuntime in interface TaskRuntimeEstimator

Parameters:

taskID - the TezTaskID of the task we are asking about

Returns:

the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

newAttemptEstimatedRuntime

public long newAttemptEstimatedRuntime()

Description copied from interface: TaskRuntimeEstimator

Estimates how long a new attempt on this task will take if we start one now

Specified by:

newAttemptEstimatedRuntime in interface TaskRuntimeEstimator

Returns:

our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.