org.apache.tez.dag.app.dag.speculation.legacy

Interface TaskRuntimeEstimator

All Known Implementing Classes:

LegacyTaskRuntimeEstimator, SimpleExponentialTaskRuntimeEstimator

public interface TaskRuntimeEstimator

Estimate the runtime for tasks of a given vertex.

Method Summary

Modifier and Type	Method and Description
long	attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)
void	contextualize(org.apache.hadoop.conf.Configuration conf, Vertex vertex)
void	enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id, long timestamp)
long	estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID id) Estimate a task attempt's total runtime.
default boolean	hasStagnatedProgress(org.apache.tez.dag.records.TezTaskAttemptID id, long timeStamp) Returns true if the estimator has no updates records for a threshold time window.
long	newAttemptEstimatedRuntime() Estimates how long a new attempt on this task will take if we start one now
long	runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID id) Computes the width of the error band of our estimate of the task runtime as returned by estimatedRuntime(TezTaskAttemptID)
long	thresholdRuntime(org.apache.tez.dag.records.TezTaskID id) Find a maximum reasonable execution wallclock time.
void	updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID taId, TaskAttemptState reportedState, long timestamp)

Method Detail

enrollAttempt

void enrollAttempt(org.apache.tez.dag.records.TezTaskAttemptID id,
                   long timestamp)

attemptEnrolledTime

long attemptEnrolledTime(org.apache.tez.dag.records.TezTaskAttemptID attemptID)

updateAttempt

void updateAttempt(org.apache.tez.dag.records.TezTaskAttemptID taId,
                   TaskAttemptState reportedState,
                   long timestamp)

contextualize

void contextualize(org.apache.hadoop.conf.Configuration conf,
                   Vertex vertex)

thresholdRuntime

long thresholdRuntime(org.apache.tez.dag.records.TezTaskID id)

Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.

Parameters:

id - the TezTaskID of the task we are asking about

Returns:

the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

estimatedRuntime

long estimatedRuntime(org.apache.tez.dag.records.TezTaskAttemptID id)

Estimate a task attempt's total runtime. Includes the time already elapsed.

Parameters:

id - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

newAttemptEstimatedRuntime

long newAttemptEstimatedRuntime()

Estimates how long a new attempt on this task will take if we start one now

Returns:

our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

runtimeEstimateVariance

long runtimeEstimateVariance(org.apache.tez.dag.records.TezTaskAttemptID id)

Computes the width of the error band of our estimate of the task runtime as returned by estimatedRuntime(TezTaskAttemptID)

Parameters:

id - the TezTaskAttemptID of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

hasStagnatedProgress

default boolean hasStagnatedProgress(org.apache.tez.dag.records.TezTaskAttemptID id,
                                     long timeStamp)

Returns true if the estimator has no updates records for a threshold time window. This helps to identify task attempts that are stalled at the beginning of execution.

Parameters:

id - the TezTaskAttemptID of the attempt we are asking about

timeStamp - the time of the report we compare with

Returns:

true if the task attempt has no progress for a given time window