org.apache.samza.validation

Class YarnJobValidationTool

java.lang.Object

org.apache.samza.validation.YarnJobValidationTool

public class YarnJobValidationTool
extends java.lang.Object

Command-line tool for validating the status of a Yarn job. It checks the job has been successfully submitted to the Yarn cluster, the status of the application attempt is running and the running container count matches the expectation. It also supports an optional MetricsValidator plugin through arguments so job metrics can be validated too using JMX. This tool can be used, for example, as an automated validation step after starting a job. When running this tool, please provide the configuration URI of job. For example: deploy/samza/bin/validate-yarn-job.sh --config job.config.loader.factory=org.apache.samza.config.loaders.PropertiesConfigLoaderFactory --config job.config.loader.properties.path=$PWD/deploy/samza/config/wikipedia-feed.properties [--metrics-validator=com.foo.bar.SomeMetricsValidator] The tool prints out the validation result in each step and throws an exception when the validation fails.

Constructor Summary

Constructors

Constructor and Description
YarnJobValidationTool(org.apache.samza.config.JobConfig config, org.apache.hadoop.yarn.client.api.YarnClient client, org.apache.samza.metrics.MetricsValidator validator)

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)
void	run()
org.apache.hadoop.yarn.api.records.ApplicationId	validateAppId()
int	validateContainerCount(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
void	validateJmxMetrics()
org.apache.hadoop.yarn.api.records.ApplicationAttemptId	validateRunningAttemptId(org.apache.hadoop.yarn.api.records.ApplicationId appId)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

YarnJobValidationTool

public YarnJobValidationTool(org.apache.samza.config.JobConfig config,
                             org.apache.hadoop.yarn.client.api.YarnClient client,
                             org.apache.samza.metrics.MetricsValidator validator)

Method Detail

run

public void run()

validateAppId

public org.apache.hadoop.yarn.api.records.ApplicationId validateAppId()
                                                               throws java.lang.Exception

Throws:

java.lang.Exception

validateRunningAttemptId

public org.apache.hadoop.yarn.api.records.ApplicationAttemptId validateRunningAttemptId(org.apache.hadoop.yarn.api.records.ApplicationId appId)
                                                                                 throws java.lang.Exception

Throws:

java.lang.Exception

validateContainerCount

public int validateContainerCount(org.apache.hadoop.yarn.api.records.ApplicationAttemptId attemptId)
                           throws java.lang.Exception

Throws:

java.lang.Exception

validateJmxMetrics

public void validateJmxMetrics()
                        throws java.lang.Exception

Throws:

java.lang.Exception

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception