org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair
Class FairSchedulerConfiguration

java.lang.Object
  org.apache.hadoop.conf.Configuration
      org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairSchedulerConfiguration

All Implemented Interfaces:

Iterable<Map.Entry<String,String>>, org.apache.hadoop.io.Writable

@InterfaceAudience.Private
@InterfaceStability.Evolving
public class FairSchedulerConfiguration
extends org.apache.hadoop.conf.Configuration

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.conf.Configuration

org.apache.hadoop.conf.Configuration.DeprecationDelta, org.apache.hadoop.conf.Configuration.IntegerRanges

Field Summary

static String	ALLOCATION_FILE
protected static String	ALLOW_UNDECLARED_POOLS Whether pools can be created that were not specified in the FS configuration file
static String	ASSIGN_MULTIPLE Whether to assign multiple containers in one check-in.
protected static String	CONTINUOUS_SCHEDULING_ENABLED Enable continuous scheduling or not.
protected static String	CONTINUOUS_SCHEDULING_SLEEP_MS Sleep time of each pass in continuous scheduling (5ms in default)
protected static String	DEFAULT_ALLOCATION_FILE
protected static boolean	DEFAULT_ALLOW_UNDECLARED_POOLS
protected static boolean	DEFAULT_ASSIGN_MULTIPLE
protected static boolean	DEFAULT_CONTINUOUS_SCHEDULING_ENABLED
protected static int	DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS
static boolean	DEFAULT_EVENT_LOG_ENABLED
protected static long	DEFAULT_LOCALITY_DELAY_NODE_MS
protected static long	DEFAULT_LOCALITY_DELAY_RACK_MS
protected static float	DEFAULT_LOCALITY_THRESHOLD
protected static float	DEFAULT_LOCALITY_THRESHOLD_NODE
protected static float	DEFAULT_LOCALITY_THRESHOLD_RACK
protected static int	DEFAULT_MAX_ASSIGN
protected static boolean	DEFAULT_PREEMPTION
protected static int	DEFAULT_PREEMPTION_INTERVAL
protected static float	DEFAULT_PREEMPTION_THRESHOLD
static int	DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB
static int	DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
protected static boolean	DEFAULT_SIZE_BASED_WEIGHT
static int	DEFAULT_UPDATE_INTERVAL_MS
protected static boolean	DEFAULT_USER_AS_DEFAULT_QUEUE
protected static int	DEFAULT_WAIT_TIME_BEFORE_KILL
protected static String	EVENT_LOG_DIR
static String	EVENT_LOG_ENABLED Whether to enable the Fair Scheduler event log
protected static String	LOCALITY_DELAY_NODE_MS Delay for node locality.
protected static String	LOCALITY_DELAY_RACK_MS Delay for rack locality.
protected static String	LOCALITY_THRESHOLD_NODE Cluster threshold for node locality.
protected static String	LOCALITY_THRESHOLD_RACK Cluster threshold for rack locality.
static org.apache.commons.logging.Log	LOG
protected static String	MAX_ASSIGN Maximum number of containers to assign on each check-in.
protected static String	PREEMPTION Whether preemption is enabled.
protected static String	PREEMPTION_INTERVAL
protected static String	PREEMPTION_THRESHOLD
static String	RM_SCHEDULER_INCREMENT_ALLOCATION_MB Increment request grant-able by the RM scheduler.
static String	RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES
protected static String	SIZE_BASED_WEIGHT Whether to give more weight to apps requiring many resources.
static String	UPDATE_INTERVAL_MS The update interval for calculating resources in FairScheduler .
protected static String	USER_AS_DEFAULT_QUEUE Whether to use the user name as the queue name (instead of "default") if the request does not specify a queue.
protected static String	WAIT_TIME_BEFORE_KILL

Constructor Summary

FairSchedulerConfiguration()

FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf)

Method Summary

boolean	getAssignMultiple()
int	getContinuousSchedulingSleepMs()
String	getEventlogDir()
org.apache.hadoop.yarn.api.records.Resource	getIncrementAllocation()
long	getLocalityDelayNodeMs()
long	getLocalityDelayRackMs()
float	getLocalityThresholdNode()
float	getLocalityThresholdRack()
int	getMaxAssign()
org.apache.hadoop.yarn.api.records.Resource	getMaximumAllocation()
org.apache.hadoop.yarn.api.records.Resource	getMinimumAllocation()
boolean	getPreemptionEnabled()
int	getPreemptionInterval()
float	getPreemptionUtilizationThreshold()
boolean	getSizeBasedWeight()
long	getUpdateInterval()
boolean	getUsePortForNodeName()
int	getWaitTimeBeforeKill()
boolean	isContinuousSchedulingEnabled()
boolean	isEventLogEnabled()
static org.apache.hadoop.yarn.api.records.Resource	parseResourceConfigValue(String val) Parses a resource config value of a form like "1024", "1024 mb", or "1024 mb, 3 vcores".

Methods inherited from class org.apache.hadoop.conf.Configuration

addDefaultResource, addDeprecation, addDeprecation, addDeprecation, addDeprecation, addDeprecations, addResource, addResource, addResource, addResource, addResource, addResource, clear, dumpConfiguration, dumpDeprecatedKeys, get, get, getBoolean, getClass, getClass, getClassByName, getClassByNameOrNull, getClasses, getClassLoader, getConfResourceAsInputStream, getConfResourceAsReader, getDouble, getEnum, getFile, getFinalParameters, getFloat, getInstances, getInt, getInts, getLocalPath, getLong, getLongBytes, getPassword, getPasswordFromConfig, getPasswordFromCredentialProviders, getPattern, getPropertySources, getProps, getRange, getRaw, getResource, getSocketAddr, getSocketAddr, getStringCollection, getStrings, getStrings, getTimeDuration, getTrimmed, getTrimmed, getTrimmedStringCollection, getTrimmedStrings, getTrimmedStrings, getValByRegex, hasWarnedDeprecation, isDeprecated, iterator, main, readFields, reloadConfiguration, set, set, setBoolean, setBooleanIfUnset, setClass, setClassLoader, setDeprecatedProperties, setDouble, setEnum, setFloat, setIfUnset, setInt, setLong, setPattern, setQuietMode, setSocketAddr, setStrings, setTimeDuration, size, toString, unset, updateConnectAddr, updateConnectAddr, write, writeXml, writeXml

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOG

public static final org.apache.commons.logging.Log LOG

RM_SCHEDULER_INCREMENT_ALLOCATION_MB

public static final String RM_SCHEDULER_INCREMENT_ALLOCATION_MB

Increment request grant-able by the RM scheduler. These properties are looked up in the yarn-site.xml

See Also:

Constant Field Values

DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB

public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_MB

See Also:

Constant Field Values

RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES

public static final String RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES

See Also:

Constant Field Values

DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES

public static final int DEFAULT_RM_SCHEDULER_INCREMENT_ALLOCATION_VCORES

See Also:

Constant Field Values

ALLOCATION_FILE

public static final String ALLOCATION_FILE

See Also:

Constant Field Values

DEFAULT_ALLOCATION_FILE

protected static final String DEFAULT_ALLOCATION_FILE

See Also:

Constant Field Values

EVENT_LOG_ENABLED

public static final String EVENT_LOG_ENABLED

Whether to enable the Fair Scheduler event log

See Also:

Constant Field Values

DEFAULT_EVENT_LOG_ENABLED

public static final boolean DEFAULT_EVENT_LOG_ENABLED

See Also:

Constant Field Values

EVENT_LOG_DIR

protected static final String EVENT_LOG_DIR

See Also:

Constant Field Values

ALLOW_UNDECLARED_POOLS

protected static final String ALLOW_UNDECLARED_POOLS

Whether pools can be created that were not specified in the FS configuration file

See Also:

Constant Field Values

DEFAULT_ALLOW_UNDECLARED_POOLS

protected static final boolean DEFAULT_ALLOW_UNDECLARED_POOLS

See Also:

Constant Field Values

USER_AS_DEFAULT_QUEUE

protected static final String USER_AS_DEFAULT_QUEUE

Whether to use the user name as the queue name (instead of "default") if the request does not specify a queue.

See Also:

Constant Field Values

DEFAULT_USER_AS_DEFAULT_QUEUE

protected static final boolean DEFAULT_USER_AS_DEFAULT_QUEUE

See Also:

Constant Field Values

DEFAULT_LOCALITY_THRESHOLD

protected static final float DEFAULT_LOCALITY_THRESHOLD

See Also:

Constant Field Values

LOCALITY_THRESHOLD_NODE

protected static final String LOCALITY_THRESHOLD_NODE

Cluster threshold for node locality.

See Also:

Constant Field Values

DEFAULT_LOCALITY_THRESHOLD_NODE

protected static final float DEFAULT_LOCALITY_THRESHOLD_NODE

See Also:

Constant Field Values

LOCALITY_THRESHOLD_RACK

protected static final String LOCALITY_THRESHOLD_RACK

Cluster threshold for rack locality.

See Also:

Constant Field Values

DEFAULT_LOCALITY_THRESHOLD_RACK

protected static final float DEFAULT_LOCALITY_THRESHOLD_RACK

See Also:

Constant Field Values

LOCALITY_DELAY_NODE_MS

protected static final String LOCALITY_DELAY_NODE_MS

Delay for node locality.

See Also:

Constant Field Values

DEFAULT_LOCALITY_DELAY_NODE_MS

protected static final long DEFAULT_LOCALITY_DELAY_NODE_MS

See Also:

Constant Field Values

LOCALITY_DELAY_RACK_MS

protected static final String LOCALITY_DELAY_RACK_MS

Delay for rack locality.

See Also:

Constant Field Values

DEFAULT_LOCALITY_DELAY_RACK_MS

protected static final long DEFAULT_LOCALITY_DELAY_RACK_MS

See Also:

Constant Field Values

CONTINUOUS_SCHEDULING_ENABLED

protected static final String CONTINUOUS_SCHEDULING_ENABLED

Enable continuous scheduling or not.

See Also:

Constant Field Values

DEFAULT_CONTINUOUS_SCHEDULING_ENABLED

protected static final boolean DEFAULT_CONTINUOUS_SCHEDULING_ENABLED

See Also:

Constant Field Values

CONTINUOUS_SCHEDULING_SLEEP_MS

protected static final String CONTINUOUS_SCHEDULING_SLEEP_MS

Sleep time of each pass in continuous scheduling (5ms in default)

See Also:

Constant Field Values

DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS

protected static final int DEFAULT_CONTINUOUS_SCHEDULING_SLEEP_MS

See Also:

Constant Field Values

PREEMPTION

protected static final String PREEMPTION

Whether preemption is enabled.

See Also:

Constant Field Values

DEFAULT_PREEMPTION

protected static final boolean DEFAULT_PREEMPTION

See Also:

Constant Field Values

PREEMPTION_THRESHOLD

protected static final String PREEMPTION_THRESHOLD

See Also:

Constant Field Values

DEFAULT_PREEMPTION_THRESHOLD

protected static final float DEFAULT_PREEMPTION_THRESHOLD

See Also:

Constant Field Values

PREEMPTION_INTERVAL

protected static final String PREEMPTION_INTERVAL

See Also:

Constant Field Values

DEFAULT_PREEMPTION_INTERVAL

protected static final int DEFAULT_PREEMPTION_INTERVAL

See Also:

Constant Field Values

WAIT_TIME_BEFORE_KILL

protected static final String WAIT_TIME_BEFORE_KILL

See Also:

Constant Field Values

DEFAULT_WAIT_TIME_BEFORE_KILL

protected static final int DEFAULT_WAIT_TIME_BEFORE_KILL

See Also:

Constant Field Values

ASSIGN_MULTIPLE

public static final String ASSIGN_MULTIPLE

Whether to assign multiple containers in one check-in.

See Also:

Constant Field Values

DEFAULT_ASSIGN_MULTIPLE

protected static final boolean DEFAULT_ASSIGN_MULTIPLE

See Also:

Constant Field Values

SIZE_BASED_WEIGHT

protected static final String SIZE_BASED_WEIGHT

Whether to give more weight to apps requiring many resources.

See Also:

Constant Field Values

DEFAULT_SIZE_BASED_WEIGHT

protected static final boolean DEFAULT_SIZE_BASED_WEIGHT

See Also:

Constant Field Values

MAX_ASSIGN

protected static final String MAX_ASSIGN

Maximum number of containers to assign on each check-in.

See Also:

Constant Field Values

DEFAULT_MAX_ASSIGN

protected static final int DEFAULT_MAX_ASSIGN

See Also:

Constant Field Values

UPDATE_INTERVAL_MS

public static final String UPDATE_INTERVAL_MS

The update interval for calculating resources in FairScheduler .

See Also:

Constant Field Values

DEFAULT_UPDATE_INTERVAL_MS

public static final int DEFAULT_UPDATE_INTERVAL_MS

See Also:

Constant Field Values

Constructor Detail

FairSchedulerConfiguration

public FairSchedulerConfiguration()

FairSchedulerConfiguration

public FairSchedulerConfiguration(org.apache.hadoop.conf.Configuration conf)

Method Detail

getMinimumAllocation

public org.apache.hadoop.yarn.api.records.Resource getMinimumAllocation()

getMaximumAllocation

public org.apache.hadoop.yarn.api.records.Resource getMaximumAllocation()

getIncrementAllocation

public org.apache.hadoop.yarn.api.records.Resource getIncrementAllocation()

getLocalityThresholdNode

public float getLocalityThresholdNode()

getLocalityThresholdRack

public float getLocalityThresholdRack()

isContinuousSchedulingEnabled

public boolean isContinuousSchedulingEnabled()

getContinuousSchedulingSleepMs

public int getContinuousSchedulingSleepMs()

getLocalityDelayNodeMs

public long getLocalityDelayNodeMs()

getLocalityDelayRackMs

public long getLocalityDelayRackMs()

getPreemptionEnabled

public boolean getPreemptionEnabled()

getPreemptionUtilizationThreshold

public float getPreemptionUtilizationThreshold()

getAssignMultiple

public boolean getAssignMultiple()

getMaxAssign

public int getMaxAssign()

getSizeBasedWeight

public boolean getSizeBasedWeight()

isEventLogEnabled

public boolean isEventLogEnabled()

getEventlogDir

public String getEventlogDir()

getPreemptionInterval

public int getPreemptionInterval()

getWaitTimeBeforeKill

public int getWaitTimeBeforeKill()

getUsePortForNodeName

public boolean getUsePortForNodeName()

parseResourceConfigValue

public static org.apache.hadoop.yarn.api.records.Resource parseResourceConfigValue(String val)
                                                                            throws AllocationConfigurationException

Parses a resource config value of a form like "1024", "1024 mb", or "1024 mb, 3 vcores". If no units are given, megabytes are assumed.

Throws:

AllocationConfigurationException

getUpdateInterval

public long getUpdateInterval()