org.apache.hadoop.yarn.server.resourcemanager.scheduler
Interface YarnScheduler

All Superinterfaces:

org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>

All Known Subinterfaces:

PreemptableResourceScheduler, ResourceScheduler

All Known Implementing Classes:

AbstractYarnScheduler, CapacityScheduler, FairScheduler, FifoScheduler

public interface YarnScheduler
extends org.apache.hadoop.yarn.event.EventHandler<SchedulerEvent>

This interface is used by the components to talk to the scheduler for allocating of resources, cleaning up resources.

Method Summary

void	addQueue(Queue newQueue) Add to the scheduler a new Queue.
Allocation	allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId, List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask, List<org.apache.hadoop.yarn.api.records.ContainerId> release, List<String> blacklistAdditions, List<String> blacklistRemovals) The main api between the ApplicationMaster and the Scheduler.
boolean	checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI, org.apache.hadoop.yarn.api.records.QueueACL acl, String queueName) Check if the user has permission to perform the operation.
org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport	getAppResourceUsageReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) Get a resource usage report from a given app attempt ID.
List<org.apache.hadoop.yarn.api.records.ApplicationAttemptId>	getAppsInQueue(String queueName) Gets the apps under a given queue
org.apache.hadoop.yarn.api.records.Resource	getClusterResource() Get the whole resource capacity of the cluster.
org.apache.hadoop.yarn.api.records.Resource	getMaximumResourceCapability() Get maximum allocatable Resource.
org.apache.hadoop.yarn.api.records.Resource	getMinimumResourceCapability() Get minimum allocatable Resource.
SchedulerNodeReport	getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId) Get node resource usage report.
int	getNumClusterNodes() Get the number of nodes available in the cluster.
Set<String>	getPlanQueues() Gets the list of names for queues managed by the Reservation System
org.apache.hadoop.yarn.api.records.QueueInfo	getQueueInfo(String queueName, boolean includeChildQueues, boolean recursive) Get queue information
List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo>	getQueueUserAclInfo() Get acls for queues for current user.
RMContainer	getRMContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Get the container for the given containerId.
QueueMetrics	getRootQueueMetrics() Get the root queue for the scheduler.
SchedulerAppReport	getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId) Get the Scheduler app for a given app attempt Id.
EnumSet<org.apache.hadoop.yarn.proto.YarnServiceProtos.SchedulerResourceTypes>	getSchedulingResourceTypes() Return a collection of the resource types that are considered when scheduling
void	killAllAppsInQueue(String queueName) Terminate all applications in the specified queue.
void	moveAllApps(String sourceQueue, String destQueue) Completely drain sourceQueue of applications, by moving all of them to destQueue.
String	moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId, String newQueue) Moves the given application to the given queue
void	removeQueue(String queueName) Remove an existing queue.
void	setEntitlement(String queue, QueueEntitlement entitlement) This method increase the entitlement for current queue (must respect invariants, e.g., no overcommit of parents, non negative, etc.).

Methods inherited from interface org.apache.hadoop.yarn.event.EventHandler

handle

Method Detail

getQueueInfo

@InterfaceAudience.Public
@InterfaceStability.Stable
org.apache.hadoop.yarn.api.records.QueueInfo getQueueInfo(String queueName,
                                                                                                             boolean includeChildQueues,
                                                                                                             boolean recursive)
                                                          throws IOException

Get queue information

Parameters:

queueName - queue name

includeChildQueues - include child queues?

recursive - get children queues?

Returns:

queue information

Throws:

IOException

getQueueUserAclInfo

@InterfaceAudience.Public
@InterfaceStability.Stable
List<org.apache.hadoop.yarn.api.records.QueueUserACLInfo> getQueueUserAclInfo()

Get acls for queues for current user.

Returns:

acls for queues for current user

getClusterResource

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Unstable
org.apache.hadoop.yarn.api.records.Resource getClusterResource()

Get the whole resource capacity of the cluster.

Returns:

the whole resource capacity of the cluster.

getMinimumResourceCapability

@InterfaceAudience.Public
@InterfaceStability.Stable
org.apache.hadoop.yarn.api.records.Resource getMinimumResourceCapability()

Get minimum allocatable Resource.

Returns:

minimum allocatable resource

getMaximumResourceCapability

@InterfaceAudience.Public
@InterfaceStability.Stable
org.apache.hadoop.yarn.api.records.Resource getMaximumResourceCapability()

Get maximum allocatable Resource.

Returns:

maximum allocatable resource

getNumClusterNodes

@InterfaceAudience.Public
@InterfaceStability.Stable
int getNumClusterNodes()

Get the number of nodes available in the cluster.

Returns:

the number of available nodes.

allocate

@InterfaceAudience.Public
@InterfaceStability.Stable
Allocation allocate(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId,
                                                                       List<org.apache.hadoop.yarn.api.records.ResourceRequest> ask,
                                                                       List<org.apache.hadoop.yarn.api.records.ContainerId> release,
                                                                       List<String> blacklistAdditions,
                                                                       List<String> blacklistRemovals)

The main api between the ApplicationMaster and the Scheduler. The ApplicationMaster is updating his future resource requirements and may release containers he doens't need.

Parameters:

appAttemptId -

ask -

release -

blacklistAdditions -

blacklistRemovals -

Returns:

the Allocation for the application

getNodeReport

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Stable
SchedulerNodeReport getNodeReport(org.apache.hadoop.yarn.api.records.NodeId nodeId)

Get node resource usage report.

Parameters:

nodeId -

Returns:

the SchedulerNodeReport for the node or null if nodeId does not point to a defined node.

getSchedulerAppInfo

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Stable
SchedulerAppReport getSchedulerAppInfo(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

Get the Scheduler app for a given app attempt Id.

Parameters:

appAttemptId - the id of the application attempt

Returns:

SchedulerApp for this given attempt.

getAppResourceUsageReport

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Evolving
org.apache.hadoop.yarn.api.records.ApplicationResourceUsageReport getAppResourceUsageReport(org.apache.hadoop.yarn.api.records.ApplicationAttemptId appAttemptId)

Get a resource usage report from a given app attempt ID.

Parameters:

appAttemptId - the id of the application attempt

Returns:

resource usage report for this given attempt

getRootQueueMetrics

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Evolving
QueueMetrics getRootQueueMetrics()

Get the root queue for the scheduler.

Returns:

the root queue for the scheduler.

checkAccess

boolean checkAccess(org.apache.hadoop.security.UserGroupInformation callerUGI,
                    org.apache.hadoop.yarn.api.records.QueueACL acl,
                    String queueName)

Check if the user has permission to perform the operation. If the user has QueueACL.ADMINISTER_QUEUE permission, this user can view/modify the applications in this queue

Parameters:

callerUGI -

acl -

queueName -

Returns:

true if the user has the permission, false otherwise

getAppsInQueue

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Stable
List<org.apache.hadoop.yarn.api.records.ApplicationAttemptId> getAppsInQueue(String queueName)

Gets the apps under a given queue

Parameters:

queueName - the name of the queue.

Returns:

a collection of app attempt ids in the given queue.

getRMContainer

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Unstable
RMContainer getRMContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Get the container for the given containerId.

Parameters:

containerId -

Returns:

the container for the given containerId.

moveApplication

@InterfaceAudience.LimitedPrivate(value="yarn")
@InterfaceStability.Evolving
String moveApplication(org.apache.hadoop.yarn.api.records.ApplicationId appId,
                                                                                    String newQueue)
                       throws org.apache.hadoop.yarn.exceptions.YarnException

Moves the given application to the given queue

Parameters:

appId -

newQueue -

Returns:

the name of the queue the application was placed into

Throws:

org.apache.hadoop.yarn.exceptions.YarnException - if the move cannot be carried out

moveAllApps

void moveAllApps(String sourceQueue,
                 String destQueue)
                 throws org.apache.hadoop.yarn.exceptions.YarnException

Completely drain sourceQueue of applications, by moving all of them to destQueue.

Parameters:

sourceQueue -

destQueue -

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

killAllAppsInQueue

void killAllAppsInQueue(String queueName)
                        throws org.apache.hadoop.yarn.exceptions.YarnException

Terminate all applications in the specified queue.

Parameters:

queueName - the name of queue to be drained

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

removeQueue

void removeQueue(String queueName)
                 throws org.apache.hadoop.yarn.exceptions.YarnException

Remove an existing queue. Implementations might limit when a queue could be removed (e.g., must have zero entitlement, and no applications running, or must be a leaf, etc..).

Parameters:

queueName - name of the queue to remove

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

addQueue

void addQueue(Queue newQueue)
              throws org.apache.hadoop.yarn.exceptions.YarnException

Add to the scheduler a new Queue. Implementations might limit what type of queues can be dynamically added (e.g., Queue must be a leaf, must be attached to existing parent, must have zero entitlement).

Parameters:

newQueue - the queue being added.

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

setEntitlement

void setEntitlement(String queue,
                    QueueEntitlement entitlement)
                    throws org.apache.hadoop.yarn.exceptions.YarnException

This method increase the entitlement for current queue (must respect invariants, e.g., no overcommit of parents, non negative, etc.). Entitlement is a general term for weights in FairScheduler, capacity for the CapacityScheduler, etc.

Parameters:

queue - the queue for which we change entitlement

entitlement - the new entitlement for the queue (capacity, maxCapacity, etc..)

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

getPlanQueues

Set<String> getPlanQueues()
                          throws org.apache.hadoop.yarn.exceptions.YarnException

Gets the list of names for queues managed by the Reservation System

Returns:

the list of queues which support reservations

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

getSchedulingResourceTypes

EnumSet<org.apache.hadoop.yarn.proto.YarnServiceProtos.SchedulerResourceTypes> getSchedulingResourceTypes()

Return a collection of the resource types that are considered when scheduling

Returns:

an EnumSet containing the resource types