org.apache.hadoop.yarn.server.nodemanager
Class ContainerExecutor

java.lang.Object
  org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

Direct Known Subclasses:

DefaultContainerExecutor, LinuxContainerExecutor

public abstract class ContainerExecutor
extends Object
implements org.apache.hadoop.conf.Configurable

Nested Class Summary

static class	ContainerExecutor.DelayedProcessKiller
static class	ContainerExecutor.ExitCode
static class	ContainerExecutor.Signal The constants for the signals.

Field Summary

static boolean	isSetsidAvailable
static org.apache.hadoop.fs.permission.FsPermission	TASK_LAUNCH_SCRIPT_PERMISSION

Constructor Summary

ContainerExecutor()

Method Summary

void	activateContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId, org.apache.hadoop.fs.Path pidFilePath) Mark the container as active
void	deactivateContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Mark the container as inactive.
abstract void	deleteAsUser(String user, org.apache.hadoop.fs.Path subDir, org.apache.hadoop.fs.Path... basedirs)
org.apache.hadoop.conf.Configuration	getConf()
protected org.apache.hadoop.fs.Path	getPidFilePath(org.apache.hadoop.yarn.api.records.ContainerId containerId) Get the pidFile of the container.
String	getProcessId(org.apache.hadoop.yarn.api.records.ContainerId containerID) Get the process-identifier for the container
protected static String[]	getRunCommand(String command, org.apache.hadoop.conf.Configuration conf) Return a command to execute the given command in OS shell.
abstract void	init() Run the executor initialization steps.
protected boolean	isContainerActive(org.apache.hadoop.yarn.api.records.ContainerId containerId) Is the container still active?
abstract int	launchContainer(Container container, org.apache.hadoop.fs.Path nmPrivateContainerScriptPath, org.apache.hadoop.fs.Path nmPrivateTokensPath, String user, String appId, org.apache.hadoop.fs.Path containerWorkDir, List<String> localDirs, List<String> logDirs) Launch the container on the node.
protected void	logOutput(String output)
void	setConf(org.apache.hadoop.conf.Configuration conf)
abstract boolean	signalContainer(String user, String pid, ContainerExecutor.Signal signal)
abstract void	startLocalizer(org.apache.hadoop.fs.Path nmPrivateContainerTokens, InetSocketAddress nmAddr, String user, String appId, String locId, List<String> localDirs, List<String> logDirs) Prepare the environment for containers in this application to execute.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TASK_LAUNCH_SCRIPT_PERMISSION

public static final org.apache.hadoop.fs.permission.FsPermission TASK_LAUNCH_SCRIPT_PERMISSION

isSetsidAvailable

public static final boolean isSetsidAvailable

Constructor Detail

ContainerExecutor

public ContainerExecutor()

Method Detail

setConf

public void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

getConf

public org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

init

public abstract void init()
                   throws IOException

Run the executor initialization steps. Verify that the necessary configs, permissions are in place.

Throws:

IOException

startLocalizer

public abstract void startLocalizer(org.apache.hadoop.fs.Path nmPrivateContainerTokens,
                                    InetSocketAddress nmAddr,
                                    String user,
                                    String appId,
                                    String locId,
                                    List<String> localDirs,
                                    List<String> logDirs)
                             throws IOException,
                                    InterruptedException

Prepare the environment for containers in this application to execute. For $x in local.dirs create $x/$user/$appId Copy $nmLocal/appTokens -> $N/$user/$appId For $rsrc in private resources Copy $rsrc -> $N/$user/filecache/[idef] For $rsrc in job resources Copy $rsrc -> $N/$user/$appId/filecache/idef

Parameters:

user - user name of application owner

appId - id of the application

nmPrivateContainerTokens - path to localized credentials, rsrc by NM

nmAddr - RPC address to contact NM

localDirs - nm-local-dirs

logDirs - nm-log-dirs

Throws:

IOException - For most application init failures

InterruptedException - If application init thread is halted by NM

launchContainer

public abstract int launchContainer(Container container,
                                    org.apache.hadoop.fs.Path nmPrivateContainerScriptPath,
                                    org.apache.hadoop.fs.Path nmPrivateTokensPath,
                                    String user,
                                    String appId,
                                    org.apache.hadoop.fs.Path containerWorkDir,
                                    List<String> localDirs,
                                    List<String> logDirs)
                             throws IOException

Launch the container on the node. This is a blocking call and returns only when the container exits.

Parameters:

container - the container to be launched

nmPrivateContainerScriptPath - the path for launch script

nmPrivateTokensPath - the path for tokens for the container

user - the user of the container

appId - the appId of the container

containerWorkDir - the work dir for the container

localDirs - nm-local-dirs to be used for this container

logDirs - nm-log-dirs to be used for this container

Returns:

the return status of the launch

Throws:

IOException

signalContainer

public abstract boolean signalContainer(String user,
                                        String pid,
                                        ContainerExecutor.Signal signal)
                                 throws IOException

Throws:

IOException

deleteAsUser

public abstract void deleteAsUser(String user,
                                  org.apache.hadoop.fs.Path subDir,
                                  org.apache.hadoop.fs.Path... basedirs)
                           throws IOException,
                                  InterruptedException

Throws:

IOException

InterruptedException

logOutput

protected void logOutput(String output)

getPidFilePath

protected org.apache.hadoop.fs.Path getPidFilePath(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Get the pidFile of the container.

Parameters:

containerId -

Returns:

the path of the pid-file for the given containerId.

getRunCommand

protected static String[] getRunCommand(String command,
                                        org.apache.hadoop.conf.Configuration conf)

Return a command to execute the given command in OS shell.

isContainerActive

protected boolean isContainerActive(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Is the container still active?

Parameters:

containerId -

Returns:

true if the container is active else false.

activateContainer

public void activateContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId,
                              org.apache.hadoop.fs.Path pidFilePath)

Mark the container as active

Parameters:

containerId - the ContainerId

pidFilePath - Path where the executor should write the pid of the launched process

deactivateContainer

public void deactivateContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Mark the container as inactive. Done iff the container is still active. Else treat it as a no-op

getProcessId

public String getProcessId(org.apache.hadoop.yarn.api.records.ContainerId containerID)

Get the process-identifier for the container

Parameters:

containerID -

Returns:

the processid of the container if it has already launched, otherwise return null