org.apache.hadoop.mapreduce.v2.app.speculate
Class LegacyTaskRuntimeEstimator

java.lang.Object
  org.apache.hadoop.mapreduce.v2.app.speculate.LegacyTaskRuntimeEstimator

All Implemented Interfaces:

TaskRuntimeEstimator

public class LegacyTaskRuntimeEstimator
extends Object

Field Summary

protected org.apache.hadoop.conf.Configuration	conf
protected AppContext	context
protected Set<Task>	doneTasks
protected Map<Job,DataStatistics>	mapperStatistics
protected Map<Job,DataStatistics>	reducerStatistics
protected Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long>	startTimes

Constructor Summary

LegacyTaskRuntimeEstimator()

Method Summary

long	attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)
void	contextualize(org.apache.hadoop.conf.Configuration conf, AppContext context)
protected DataStatistics	dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)
void	enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)
long	estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id) Estimates how long a new attempt on this task will take if we start one now
long	estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) Estimate a task attempt's total runtime.
long	runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID) Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)
long	thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID) Find a maximum reasonable execution wallclock time.
void	updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status, long timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

context

protected AppContext context

startTimes

protected final Map<org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId,Long> startTimes

mapperStatistics

protected final Map<Job,DataStatistics> mapperStatistics

reducerStatistics

protected final Map<Job,DataStatistics> reducerStatistics

doneTasks

protected final Set<Task> doneTasks

Constructor Detail

LegacyTaskRuntimeEstimator

public LegacyTaskRuntimeEstimator()

Method Detail

updateAttempt

public void updateAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
                          long timestamp)

Specified by:

updateAttempt in interface TaskRuntimeEstimator

estimatedRuntime

public long estimatedRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Description copied from interface: TaskRuntimeEstimator

Estimate a task attempt's total runtime. Includes the time already elapsed.

Parameters:

attemptID - the TaskAttemptId of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

runtimeEstimateVariance

public long runtimeEstimateVariance(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Description copied from interface: TaskRuntimeEstimator

Computes the width of the error band of our estimate of the task runtime as returned by TaskRuntimeEstimator.estimatedRuntime(TaskAttemptId)

Parameters:

attemptID - the TaskAttemptId of the attempt we are asking about

Returns:

our best estimate of the attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.

enrollAttempt

public void enrollAttempt(TaskAttemptStatusUpdateEvent.TaskAttemptStatus status,
                          long timestamp)

Specified by:

enrollAttempt in interface TaskRuntimeEstimator

attemptEnrolledTime

public long attemptEnrolledTime(org.apache.hadoop.mapreduce.v2.api.records.TaskAttemptId attemptID)

Specified by:

attemptEnrolledTime in interface TaskRuntimeEstimator

contextualize

public void contextualize(org.apache.hadoop.conf.Configuration conf,
                          AppContext context)

Specified by:

contextualize in interface TaskRuntimeEstimator

dataStatisticsForTask

protected DataStatistics dataStatisticsForTask(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)

thresholdRuntime

public long thresholdRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId taskID)

Description copied from interface: TaskRuntimeEstimator

Find a maximum reasonable execution wallclock time. Includes the time already elapsed. Find a maximum reasonable execution time. Includes the time already elapsed. If the projected total execution time for this task ever exceeds its reasonable execution time, we may speculate it.

Specified by:

thresholdRuntime in interface TaskRuntimeEstimator

Parameters:

taskID - the TaskId of the task we are asking about

Returns:

the task's maximum reasonable runtime, or MAX_VALUE if we don't have enough information to rule out any runtime, however long.

estimatedNewAttemptRuntime

public long estimatedNewAttemptRuntime(org.apache.hadoop.mapreduce.v2.api.records.TaskId id)

Description copied from interface: TaskRuntimeEstimator

Estimates how long a new attempt on this task will take if we start one now

Specified by:

estimatedNewAttemptRuntime in interface TaskRuntimeEstimator

Parameters:

id - the TaskId of the task we are asking about

Returns:

our best estimate of a new attempt's runtime, or -1 if we don't have enough information yet to produce an estimate.