A C D F G H I L M N O P Q R S T V Y

A

ALLOCATOR_SLEEP_MS - Static variable in class org.apache.samza.config.YarnConfig

Sleep interval for the allocator thread in milliseconds

AM_CONTAINER_LABEL - Static variable in class org.apache.samza.config.YarnConfig

Label to request from YARN for running the AM

AM_CONTAINER_MAX_CPU_CORES - Static variable in class org.apache.samza.config.YarnConfig

Number of CPU cores to request from YARN for running the AM

AM_CONTAINER_MAX_MEMORY_MB - Static variable in class org.apache.samza.config.YarnConfig

Memory, in megabytes, to request from YARN for running the AM

AM_JAVA_HOME - Static variable in class org.apache.samza.config.YarnConfig

(Optional) JAVA_HOME path for Samza AM

AM_JMX_ENABLED - Static variable in class org.apache.samza.config.YarnConfig

Determines whether a JMX server should be started on the AM Default: true

AM_JVM_OPTIONS - Static variable in class org.apache.samza.config.YarnConfig

(Optional) JVM options to include in the command line when executing the AM

AM_POLL_INTERVAL_MS - Static variable in class org.apache.samza.config.YarnConfig

Determines the interval for the Heartbeat between the AM and the Yarn RM

amContainerId - Variable in class org.apache.samza.job.yarn.YarnAppState

Id of the AM container (as allocated by the RM) Used for displaying in the AM UI.

appAttemptId - Variable in class org.apache.samza.job.yarn.YarnAppState

Application Attempt Id as provided by Yarn Used for displaying in the AM UI.

ApplicationMasterRestClient - Class in org.apache.samza.webapp

Client for the ApplicationMasterRestServlet.

ApplicationMasterRestClient(CloseableHttpClient, String, int) - Constructor for class org.apache.samza.webapp.ApplicationMasterRestClient

C

cancelResourceRequest(SamzaResourceRequest) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Remove a previously submitted resource request.

close() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

CONTAINER_LABEL - Static variable in class org.apache.samza.config.YarnConfig

Label to request from YARN for containers

CONTAINER_REQUEST_TIMEOUT_MS - Static variable in class org.apache.samza.config.YarnConfig

Number of milliseconds before a container request is considered to have to expired

coordinatorUrl - Variable in class org.apache.samza.job.yarn.YarnAppState

Job Coordinator URL Usage in SamzaYarnAppMasterService & YarnContainerRunner

D

DEFAULT_SAMZA_APPLICATION_TYPE - Static variable in class org.apache.samza.config.YarnConfig

doGet(HttpServletRequest, HttpServletResponse) - Method in class org.apache.samza.webapp.YarnContainerHeartbeatServlet

F

failedContainersStatus - Variable in class org.apache.samza.job.yarn.YarnAppState

FileSystemImplConfig - Class in org.apache.samza.job.yarn

FileSystemImplConfig is intended to manage the Samza config for fs.<scheme>impl.

FileSystemImplConfig(Config) - Constructor for class org.apache.samza.job.yarn.FileSystemImplConfig

G

getAllFaultDomains() - Method in class org.apache.samza.job.yarn.YarnFaultDomainManager

This method returns all the last cached rack values in a cluster, for all hosts that are healthy, up and running.

getAllocatorSleepTime() - Method in class org.apache.samza.config.YarnConfig

getAMContainerLabel() - Method in class org.apache.samza.config.YarnConfig

getAMContainerMaxCpuCores() - Method in class org.apache.samza.config.YarnConfig

getAMContainerMaxMemoryMb() - Method in class org.apache.samza.config.YarnConfig

getAMJavaHome() - Method in class org.apache.samza.config.YarnConfig

getAmOpts() - Method in class org.apache.samza.config.YarnConfig

getAMPollIntervalMs() - Method in class org.apache.samza.config.YarnConfig

getAmState() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

getClusterResourceManager(ClusterResourceManager.Callback, SamzaApplicationState) - Method in class org.apache.samza.job.yarn.YarnResourceManagerFactory

getConfig() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

getContainerLabel() - Method in class org.apache.samza.config.YarnConfig

getContainerRequestTimeout() - Method in class org.apache.samza.config.YarnConfig

getFaultDomainManager(Config, MetricsRegistry) - Method in class org.apache.samza.job.yarn.YarnFaultDomainManagerFactory

getFaultDomainsForHost(String) - Method in class org.apache.samza.job.yarn.YarnFaultDomainManager

This method returns the rack a particular host resides on based on the internal cache.

getJmxServerEnabled() - Method in class org.apache.samza.config.YarnConfig

getMetrics() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

getPackagePath() - Method in class org.apache.samza.config.YarnConfig

getProgress() - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

getQueueName() - Method in class org.apache.samza.config.YarnConfig

getResourceLocalName(String) - Method in class org.apache.samza.job.yarn.LocalizerResourceConfig

getResourceLocalType(String) - Method in class org.apache.samza.job.yarn.LocalizerResourceConfig

getResourceLocalVisibility(String) - Method in class org.apache.samza.job.yarn.LocalizerResourceConfig

getResourceMap() - Method in class org.apache.samza.job.yarn.LocalizerResourceMapper

getResourceNames() - Method in class org.apache.samza.job.yarn.LocalizerResourceConfig

getResourcePath(String) - Method in class org.apache.samza.job.yarn.LocalizerResourceConfig

getSchemeConfig(String) - Method in class org.apache.samza.job.yarn.FileSystemImplConfig

Get the config subset for fs.<scheme>.impl It can include config for fs.<scheme>.impl and additional config for the subKeys fs.<scheme>.impl.* from the configuration e.g.

getSchemes() - Method in class org.apache.samza.job.yarn.FileSystemImplConfig

Get all schemes

getTaskContext() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

getYarnApplicationAcls() - Method in class org.apache.samza.config.YarnConfig

Helper function to get all application acls

getYarnApplicationModifyAcl() - Method in class org.apache.samza.config.YarnConfig

getYarnApplicationType() - Method in class org.apache.samza.config.YarnConfig

getYarnApplicationViewAcl() - Method in class org.apache.samza.config.YarnConfig

getYarnCredentialsFile() - Method in class org.apache.samza.config.YarnConfig

getYarnJobStagingDirectory() - Method in class org.apache.samza.config.YarnConfig

getYarnKerberosKeytab() - Method in class org.apache.samza.config.YarnConfig

getYarnKerberosPrincipal() - Method in class org.apache.samza.config.YarnConfig

getYarnTokenRenewalIntervalSeconds() - Method in class org.apache.samza.config.YarnConfig

H

hasSameFaultDomains(String, String) - Method in class org.apache.samza.job.yarn.YarnFaultDomainManager

This method checks if the two hostnames provided reside on the same rack.

I

isResourceExpired(SamzaResource) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

L

launchStreamProcessor(SamzaResource, CommandBuilder) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Requests the launch of a StreamProcessor with the specified ID on the resource

LocalizerResourceConfig - Class in org.apache.samza.job.yarn

LocalizerResourceConfig is intended to manage/fetch the config values for the yarn localizer resource(s) from the configuration.

LocalizerResourceConfig(Config) - Constructor for class org.apache.samza.job.yarn.LocalizerResourceConfig

LocalizerResourceException - Exception in org.apache.samza.job.yarn

LocalizerResourceException() - Constructor for exception org.apache.samza.job.yarn.LocalizerResourceException

Constructs an LocalizerResourceException with null as its error detail message.

LocalizerResourceException(String) - Constructor for exception org.apache.samza.job.yarn.LocalizerResourceException

Constructs an LocalizerResourceException with the specified detail message.

LocalizerResourceException(String, Throwable) - Constructor for exception org.apache.samza.job.yarn.LocalizerResourceException

Constructs an LocalizerResourceException with the specified detail message and cause.

LocalizerResourceException(Throwable) - Constructor for exception org.apache.samza.job.yarn.LocalizerResourceException

Constructs an LocalizerResourceException with the specified cause and a detail message of (cause==null ? null : cause.toString())

LocalizerResourceMapper - Class in org.apache.samza.job.yarn

A universal approach to generate local resource map which can be put in ContainerLaunchContext directly

LocalizerResourceMapper(LocalizerResourceConfig, YarnConfiguration) - Constructor for class org.apache.samza.job.yarn.LocalizerResourceMapper

M

main(String[]) - Static method in class org.apache.samza.validation.YarnJobValidationTool

N

nodeHost - Variable in class org.apache.samza.job.yarn.YarnAppState

Host name of the NM on which the AM is running Used for displaying in the AM UI.

nodeHttpPort - Variable in class org.apache.samza.job.yarn.YarnAppState

Http port of the NM on which the AM is running Used for displaying in the AM UI.

nodePort - Variable in class org.apache.samza.job.yarn.YarnAppState

NM port on which the AM is running Used for displaying in the AM UI.

O

onContainersAllocated(List<Container>) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Callback invoked from Yarn when containers are allocated.

onContainersCompleted(List<ContainerStatus>) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Callback invoked from Yarn when containers complete.

onContainerStarted(ContainerId, Map<String, ByteBuffer>) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onContainerStatusReceived(ContainerId, ContainerStatus) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onContainerStopped(ContainerId) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onError(Throwable) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Callback invoked when there is an error in the Yarn client.

onGetContainerStatusError(ContainerId, Throwable) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onNodesUpdated(List<NodeReport>) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onShutdownRequest() - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onStartContainerError(ContainerId, Throwable) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

onStopContainerError(ContainerId, Throwable) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

org.apache.samza.config - package org.apache.samza.config

org.apache.samza.job.yarn - package org.apache.samza.job.yarn

org.apache.samza.validation - package org.apache.samza.validation

org.apache.samza.webapp - package org.apache.samza.webapp

P

PACKAGE_PATH - Static variable in class org.apache.samza.config.YarnConfig

(Required) URL from which the job package can be downloaded

pendingProcessors - Variable in class org.apache.samza.job.yarn.YarnAppState

Q

QUEUE_NAME - Static variable in class org.apache.samza.config.YarnConfig

Name of YARN queue to run jobs on

R

releaseResources(SamzaResource) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Requests the YarnContainerManager to release a resource.

requestResources(SamzaResourceRequest) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Request resources for running container processes.

rpcUrl - Variable in class org.apache.samza.job.yarn.YarnAppState

URL of the ApplicationMasterRestServlet

run() - Method in class org.apache.samza.validation.YarnJobValidationTool

runningProcessors - Variable in class org.apache.samza.job.yarn.YarnAppState

/** State indicating whether the job is healthy or not Modified by both the AMRMCallbackThread and the ContainerAllocator thread

runProcessor(String, Container, CommandBuilder) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Runs a process as specified by the command builder on the container.

S

start() - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Starts the YarnClusterResourceManager and initialize all its sub-systems.

stop(SamzaApplicationState.SamzaAppStatus) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

Stops the YarnContainerManager and all its sub-components.

stopStreamProcessor(SamzaResource) - Method in class org.apache.samza.job.yarn.YarnClusterResourceManager

T

taskId - Variable in class org.apache.samza.job.yarn.YarnAppState

Task Id of the AM Used for displaying in the AM UI.

toString() - Method in class org.apache.samza.job.yarn.YarnAppState

toString() - Method in class org.apache.samza.webapp.ApplicationMasterRestClient

trackingUrl - Variable in class org.apache.samza.job.yarn.YarnAppState

URL of the ApplicationMasterWebServlet

V

validateAppId() - Method in class org.apache.samza.validation.YarnJobValidationTool

validateContainerCount(ApplicationAttemptId) - Method in class org.apache.samza.validation.YarnJobValidationTool

validateJmxMetrics() - Method in class org.apache.samza.validation.YarnJobValidationTool

validateRunningAttemptId(ApplicationId) - Method in class org.apache.samza.validation.YarnJobValidationTool

Y

YARN_APPLICATION_MODIFY_ACL - Static variable in class org.apache.samza.config.YarnConfig

For secured YARN cluster only.

YARN_APPLICATION_TYPE - Static variable in class org.apache.samza.config.YarnConfig

Type of Samza application submitted to YARN Can be used to distinguish between different job types/APIs (eg: HighLevel, LowLevel, SQL, Beam etc)

YARN_APPLICATION_VIEW_ACL - Static variable in class org.apache.samza.config.YarnConfig

For secured YARN cluster only.

YARN_CREDENTIALS_FILE - Static variable in class org.apache.samza.config.YarnConfig

The location on HDFS to store the credentials file

YARN_JOB_STAGING_DIRECTORY - Static variable in class org.apache.samza.config.YarnConfig

The staging directory on HDFS for the job

YARN_KERBEROS_KEYTAB - Static variable in class org.apache.samza.config.YarnConfig

Key tab used to log in on a Kerberized secure cluster

YARN_KERBEROS_PRINCIPAL - Static variable in class org.apache.samza.config.YarnConfig

Principal used to log in on a Kerberized secure cluster

YARN_TOKEN_RENEWAL_INTERVAL_SECONDS - Static variable in class org.apache.samza.config.YarnConfig

Interval in seconds to renew a delegation token in Kerberized secure cluster

YarnAppState - Class in org.apache.samza.job.yarn

YarnAppState encapsulates Yarn specific state variables that are Yarn specific.

YarnAppState(int, ContainerId, String, int, int) - Constructor for class org.apache.samza.job.yarn.YarnAppState

YarnClusterResourceManager - Class in org.apache.samza.job.yarn

An YarnClusterResourceManager implements a ClusterResourceManager using Yarn as the underlying resource manager.

YarnClusterResourceManager(Config, JobModelManager, ClusterResourceManager.Callback, SamzaApplicationState) - Constructor for class org.apache.samza.job.yarn.YarnClusterResourceManager

Creates an YarnClusterResourceManager from config, a jobModelReader and a callback.

YarnConfig - Class in org.apache.samza.config

YarnConfig(Config) - Constructor for class org.apache.samza.config.YarnConfig

YarnContainerHeartbeatServlet - Class in org.apache.samza.webapp

Responds to heartbeat requests from the containers with a ContainerHeartbeatResponse.

YarnContainerHeartbeatServlet(YarnAppState, ReadableMetricsRegistry) - Constructor for class org.apache.samza.webapp.YarnContainerHeartbeatServlet

YarnFaultDomainManager - Class in org.apache.samza.job.yarn

This class functionality works with the assumption that the job.standbytasks.replication.factor is 2.

YarnFaultDomainManager(MetricsRegistry) - Constructor for class org.apache.samza.job.yarn.YarnFaultDomainManager

YarnFaultDomainManagerFactory - Class in org.apache.samza.job.yarn

A factory to build a YarnFaultDomainManager.

YarnFaultDomainManagerFactory() - Constructor for class org.apache.samza.job.yarn.YarnFaultDomainManagerFactory

YarnJobValidationTool - Class in org.apache.samza.validation

Command-line tool for validating the status of a Yarn job.

YarnJobValidationTool(JobConfig, YarnClient, MetricsValidator) - Constructor for class org.apache.samza.validation.YarnJobValidationTool

YarnResourceManagerFactory - Class in org.apache.samza.job.yarn

A YarnContainerProcessManagerFactory returns an implementation of a ClusterResourceManager for Yarn.

YarnResourceManagerFactory() - Constructor for class org.apache.samza.job.yarn.YarnResourceManagerFactory

A C D F G H I L M N O P Q R S T V Y