org.apache.hadoop.yarn.server.resourcemanager.scheduler

Class AbstractYarnScheduler<T extends SchedulerApplicationAttempt,N extends SchedulerNode>

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.server.resourcemanager.scheduler.AbstractYarnScheduler<T,N>

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service, org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>, Recoverable, ResourceScheduler, YarnScheduler

Direct Known Subclasses:

CapacityScheduler, FairScheduler, FifoScheduler

public abstract class AbstractYarnScheduler<T extends SchedulerApplicationAttempt,N extends SchedulerNode>
extends org.apache.hadoop.service.AbstractService
implements ResourceScheduler

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<org.apache.hadoop.yarn.api.records.ApplicationId,SchedulerApplication<T>>	applications
protected org.apache.hadoop.yarn.api.records.Resource	clusterResource
protected static Allocation	EMPTY_ALLOCATION
protected static List<org.apache.hadoop.yarn.api.records.Container>	EMPTY_CONTAINER_LIST
protected org.apache.hadoop.yarn.api.records.Resource	maximumAllocation
protected org.apache.hadoop.yarn.api.records.Resource	minimumAllocation
protected Map<org.apache.hadoop.yarn.api.records.NodeId,N>	nodes
protected RMContext	rmContext

Constructor Summary

Constructors

Constructor and Description
AbstractYarnScheduler(String name) Construct the service.

Method Summary

Methods

Modifier and Type	Method and Description
T	getApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId)
org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport	getAppResourceUsageReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) Get a resource usage report from a given app attempt ID.
org.apache.hadoop.yarn.api.records.Resource	getClusterResource() Get the whole resource capacity of the cluster.
T	getCurrentAttemptForContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)
org.apache.hadoop.yarn.api.records.Resource	getMaximumResourceCapability() Get maximum allocatable Resource.
org.apache.hadoop.yarn.api.records.Resource	getMinimumResourceCapability() Get minimum allocatable Resource.
SchedulerNodeReport	getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId) Get node resource usage report.
RMContainer	getRMContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Get the container for the given containerId.
SchedulerAppReport	getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) Get the Scheduler app for a given app attempt Id.
Map<org.apache.hadoop.yarn.api.records.ApplicationId,SchedulerApplication<T>>	getSchedulerApplications()
SchedulerNode	getSchedulerNode(org.apache.hadoop.yarn.api.records.NodeId nodeId)
List<org.apache.hadoop.yarn.api.records.Container>	getTransferredContainers(org.apache.hadoop.yarn.api.records.ApplicationAttemptId currentAttempt)
String	moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId, String newQueue) Moves the given application to the given queue
void	recoverContainersOnNode(List<org.apache.hadoop.yarn.server.api.protocolrecords.NMContainerStatus> containerReports, RMNode nm)
protected void	recoverResourceRequestForContainer(RMContainer rmContainer) Recover resource request back from RMContainer when a container is preempted before AM pulled the same.

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, serviceInit, serviceStart, serviceStop, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hadoop.yarn.server.resourcemanager.scheduler.ResourceScheduler

reinitialize, setRMContext

Methods inherited from interface org.apache.hadoop.yarn.server.resourcemanager.scheduler.YarnScheduler

allocate, checkAccess, getAppsInQueue, getNumClusterNodes, getQueueInfo, getQueueUserAclInfo, getRootQueueMetrics

Methods inherited from interface org.apache.hadoop.yarn.event.EventHandler

handle

Methods inherited from interface org.apache.hadoop.yarn.server.resourcemanager.recovery.Recoverable

recover

Field Detail

nodes

protected Map<org.apache.hadoop.yarn.api.records.NodeId,N extends SchedulerNode> nodes

clusterResource

protected org.apache.hadoop.yarn.api.records.Resource clusterResource

minimumAllocation

protected org.apache.hadoop.yarn.api.records.Resource minimumAllocation

maximumAllocation

protected org.apache.hadoop.yarn.api.records.Resource maximumAllocation

rmContext

protected RMContext rmContext

applications

protected Map<org.apache.hadoop.yarn.api.records.ApplicationId,SchedulerApplication<T extends SchedulerApplicationAttempt>> applications

EMPTY_CONTAINER_LIST

protected static final List<org.apache.hadoop.yarn.api.records.Container> EMPTY_CONTAINER_LIST

EMPTY_ALLOCATION

protected static final Allocation EMPTY_ALLOCATION

Constructor Detail

AbstractYarnScheduler

public AbstractYarnScheduler(String name)

Construct the service.

Parameters:

name - service name

Method Detail

getTransferredContainers

public List<org.apache.hadoop.yarn.api.records.Container> getTransferredContainers(org.apache.hadoop.yarn.api.records.ApplicationAttemptId currentAttempt)

getSchedulerApplications

public Map<org.apache.hadoop.yarn.api.records.ApplicationId,SchedulerApplication<T>> getSchedulerApplications()

getClusterResource

public org.apache.hadoop.yarn.api.records.Resource getClusterResource()

Description copied from interface: YarnScheduler

Get the whole resource capacity of the cluster.

Specified by:

getClusterResource in interface YarnScheduler

Returns:

the whole resource capacity of the cluster.

getMinimumResourceCapability

public org.apache.hadoop.yarn.api.records.Resource getMinimumResourceCapability()

Description copied from interface: YarnScheduler

Get minimum allocatable Resource.

Specified by:

getMinimumResourceCapability in interface YarnScheduler

Returns:

minimum allocatable resource

getMaximumResourceCapability

public org.apache.hadoop.yarn.api.records.Resource getMaximumResourceCapability()

Description copied from interface: YarnScheduler

Get maximum allocatable Resource.

Specified by:

getMaximumResourceCapability in interface YarnScheduler

Returns:

maximum allocatable resource

getApplicationAttempt

public T getApplicationAttempt(org.apache.hadoop.yarn.api.records.ApplicationAttemptId applicationAttemptId)

getSchedulerAppInfo

public SchedulerAppReport getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

Description copied from interface: YarnScheduler

Get the Scheduler app for a given app attempt Id.

Specified by:

getSchedulerAppInfo in interface YarnScheduler

Parameters:

appAttemptId - the id of the application attempt

Returns:

SchedulerApp for this given attempt.

getAppResourceUsageReport

public org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport getAppResourceUsageReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

Description copied from interface: YarnScheduler

Get a resource usage report from a given app attempt ID.

Specified by:

getAppResourceUsageReport in interface YarnScheduler

Parameters:

appAttemptId - the id of the application attempt

Returns:

resource usage report for this given attempt

getCurrentAttemptForContainer

public T getCurrentAttemptForContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

getRMContainer

public RMContainer getRMContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Description copied from interface: YarnScheduler

Get the container for the given containerId.

Specified by:

getRMContainer in interface YarnScheduler

Returns:

the container for the given containerId.

getNodeReport

public SchedulerNodeReport getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId)

Description copied from interface: YarnScheduler

Get node resource usage report.

Specified by:

getNodeReport in interface YarnScheduler

Returns:

the SchedulerNodeReport for the node or null if nodeId does not point to a defined node.

moveApplication

public String moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                     String newQueue)
                       throws org.apache.hadoop.yarn.exceptions.YarnException

Description copied from interface: YarnScheduler

Moves the given application to the given queue

Specified by:

moveApplication in interface YarnScheduler

Returns:

the name of the queue the application was placed into

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if the move cannot be carried out

recoverContainersOnNode

public void recoverContainersOnNode(List<org.apache.hadoop.yarn.server.api.protocolrecords.NMContainerStatus> containerReports,
                           RMNode nm)

recoverResourceRequestForContainer

protected void recoverResourceRequestForContainer(RMContainer rmContainer)

Recover resource request back from RMContainer when a container is preempted before AM pulled the same. If container is pulled by AM, then RMContainer will not have resource request to recover.

Parameters:

rmContainer -

getSchedulerNode

public SchedulerNode getSchedulerNode(org.apache.hadoop.yarn.api.records.NodeId nodeId)