org.apache.hadoop.tools.rumen

Class ZombieJob

java.lang.Object

org.apache.hadoop.tools.rumen.ZombieJob

All Implemented Interfaces:

JobStory

public class ZombieJob
extends Object
implements JobStory

ZombieJob is a layer above LoggedJob raw JSON objects. Each ZombieJob object represents a job in job history. For everything that exists in job history, contents are returned unchanged faithfully. To get input splits of a non-exist task, a non-exist task attempt, or an ill-formed task attempt, proper objects are made up from statistical sketches.

Constructor Summary

Constructors

Constructor and Description
ZombieJob(LoggedJob job, ClusterStory cluster) This constructor creates a ZombieJob with the same semantics as the LoggedJob passed in this parameter
ZombieJob(LoggedJob job, ClusterStory cluster, long seed) This constructor creates a ZombieJob with the same semantics as the LoggedJob passed in this parameter

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.InputSplit[]	getInputSplits() Get the input splits for the job.
org.apache.hadoop.mapred.JobConf	getJobConf() Get the JobConf for the job.
org.apache.hadoop.mapreduce.JobID	getJobID() Get the job ID
LoggedJob	getLoggedJob() Get the underlining LoggedJob object read directly from the trace.
TaskAttemptInfo	getMapTaskAttemptInfoAdjusted(int taskNumber, int taskAttemptNumber, int locality) Get a TaskAttemptInfo with a TaskAttemptID associated with taskType, taskNumber, and taskAttemptNumber.
String	getName() Get the job name.
int	getNumberMaps() Get the number of maps in the JobStory.
int	getNumberReduces() Get the number of reduce in the JobStory.
int	getNumLoggedMaps() Getting the number of map tasks that are actually logged in the trace.
int	getNumLoggedReduces() Getting the number of reduce tasks that are actually logged in the trace.
Pre21JobHistoryConstants.Values	getOutcome() Get the outcome of the job execution.
String	getQueueName() Get the queue where the job is submitted.
long	getSubmissionTime() Get the job submission time.
TaskAttemptInfo	getTaskAttemptInfo(org.apache.hadoop.mapreduce.TaskType taskType, int taskNumber, int taskAttemptNumber) Get a TaskAttemptInfo with a TaskAttemptID associated with taskType, taskNumber, and taskAttemptNumber.
TaskInfo	getTaskInfo(org.apache.hadoop.mapreduce.TaskType taskType, int taskNumber) Get TaskInfo for a given task.
String	getUser() Get the user who ran the job.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ZombieJob

public ZombieJob(LoggedJob job,
         ClusterStory cluster,
         long seed)

This constructor creates a ZombieJob with the same semantics as the LoggedJob passed in this parameter

Parameters:

job - The dead job this ZombieJob instance is based on.

cluster - The cluster topology where the dead job ran on. This argument can be null if we do not have knowledge of the cluster topology.

seed - Seed for the random number generator for filling in information not available from the ZombieJob.

ZombieJob

public ZombieJob(LoggedJob job,
         ClusterStory cluster)

This constructor creates a ZombieJob with the same semantics as the LoggedJob passed in this parameter

Parameters:

job - The dead job this ZombieJob instance is based on.

cluster - The cluster topology where the dead job ran on. This argument can be null if we do not have knowledge of the cluster topology.

Method Detail

getJobConf

public org.apache.hadoop.mapred.JobConf getJobConf()

Description copied from interface: JobStory

Get the JobConf for the job.

Specified by:

getJobConf in interface JobStory

Returns:

the JobConf for the job

getInputSplits

public org.apache.hadoop.mapreduce.InputSplit[] getInputSplits()

Description copied from interface: JobStory

Get the input splits for the job.

Specified by:

getInputSplits in interface JobStory

Returns:

the input splits for the job

getName

public String getName()

Description copied from interface: JobStory

Get the job name.

Specified by:

getName in interface JobStory

Returns:

the job name

getJobID

public org.apache.hadoop.mapreduce.JobID getJobID()

Description copied from interface: JobStory

Get the job ID

Specified by:

getJobID in interface JobStory

Returns:

the job ID

getNumberMaps

public int getNumberMaps()

Description copied from interface: JobStory

Get the number of maps in the JobStory.

Specified by:

getNumberMaps in interface JobStory

Returns:

the number of maps in the Job

getNumberReduces

public int getNumberReduces()

Description copied from interface: JobStory

Get the number of reduce in the JobStory.

Specified by:

getNumberReduces in interface JobStory

Returns:

the number of reduces in the Job

getOutcome

public Pre21JobHistoryConstants.Values getOutcome()

Description copied from interface: JobStory

Get the outcome of the job execution.

Specified by:

getOutcome in interface JobStory

Returns:

The outcome of the job execution.

getSubmissionTime

public long getSubmissionTime()

Description copied from interface: JobStory

Get the job submission time.

Specified by:

getSubmissionTime in interface JobStory

Returns:

the job submission time

getQueueName

public String getQueueName()

Description copied from interface: JobStory

Get the queue where the job is submitted.

Specified by:

getQueueName in interface JobStory

Returns:

the queue where the job is submitted.

getNumLoggedMaps

public int getNumLoggedMaps()

Getting the number of map tasks that are actually logged in the trace.

Returns:

The number of map tasks that are actually logged in the trace.

getNumLoggedReduces

public int getNumLoggedReduces()

Getting the number of reduce tasks that are actually logged in the trace.

Returns:

The number of map tasks that are actually logged in the trace.

getUser

public String getUser()

Description copied from interface: JobStory

Get the user who ran the job.

Specified by:

getUser in interface JobStory

Returns:

the user who ran the job

getLoggedJob

public LoggedJob getLoggedJob()

Get the underlining LoggedJob object read directly from the trace. This is mainly for debugging.

Returns:

the underlining LoggedJob object

getTaskAttemptInfo

public TaskAttemptInfo getTaskAttemptInfo(org.apache.hadoop.mapreduce.TaskType taskType,
                                 int taskNumber,
                                 int taskAttemptNumber)

Get a TaskAttemptInfo with a TaskAttemptID associated with taskType, taskNumber, and taskAttemptNumber. This function does not care about locality, and follows the following decision logic: 1. Make up a TaskAttemptInfo if the task attempt is missing in trace, 2. Make up a TaskAttemptInfo if the task attempt has a KILLED final status in trace, 3. Otherwise (final state is SUCCEEDED or FAILED), construct the TaskAttemptInfo from the trace.

Specified by:

getTaskAttemptInfo in interface JobStory

Parameters:

taskType - TaskType of the task-attempt

taskNumber - Partition number of the task-attempt

taskAttemptNumber - Attempt number of the task

Returns:

the TaskAttemptInfo for the given task-attempt

getTaskInfo

public TaskInfo getTaskInfo(org.apache.hadoop.mapreduce.TaskType taskType,
                   int taskNumber)

Description copied from interface: JobStory

Get TaskInfo for a given task.

Specified by:

getTaskInfo in interface JobStory

Parameters:

taskType - TaskType of the task

taskNumber - Partition number of the task

Returns:

the TaskInfo for the given task

getMapTaskAttemptInfoAdjusted

public TaskAttemptInfo getMapTaskAttemptInfoAdjusted(int taskNumber,
                                            int taskAttemptNumber,
                                            int locality)

Get a TaskAttemptInfo with a TaskAttemptID associated with taskType, taskNumber, and taskAttemptNumber. This function considers locality, and follows the following decision logic: 1. Make up a TaskAttemptInfo if the task attempt is missing in trace, 2. Make up a TaskAttemptInfo if the task attempt has a KILLED final status in trace, 3. If final state is FAILED, construct a TaskAttemptInfo from the trace, without considering locality. 4. If final state is SUCCEEDED, construct a TaskAttemptInfo from the trace, with runtime scaled according to locality in simulation and locality in trace.

Specified by:

getMapTaskAttemptInfoAdjusted in interface JobStory

Parameters:

taskNumber - Partition number of the task-attempt

taskAttemptNumber - Attempt number of the task

locality - Data locality of the task as scheduled in simulation

Returns:

the TaskAttemptInfo for the given task-attempt