org.apache.tez.examples

Class CartesianProduct

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.tez.examples.TezExampleBase

org.apache.tez.examples.CartesianProduct

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.util.Tool

public class CartesianProduct
extends TezExampleBase

This DAG does cartesian product of two text inputs and then filters results according to the third text input. V1 V2 V3 \ | / CP\ CP| / Broadcast \ | / Vertex 4 Vertex 1~3 are tokenizers and each of them tokenizes input from one directory. In partitioned case, CustomPartitioner separates tokens into 2 partitions according to the parity of token's first char. Vertex 4 does cartesian product of input from vertex1 and vertex2, and generates KV pairs where keys are vertex 1 tokens and values are vertex 2 tokens. Then vertex 4 outputs KV pairs whose keys appears in vertex 3 tokens.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	CartesianProduct.CustomPartitioner
static class	CartesianProduct.JoinProcessor
static class	CartesianProduct.TokenProcessor

Field Summary

Fields inherited from class org.apache.tez.examples.TezExampleBase

COUNTER_LOG, DISABLE_SPLIT_GROUPING, GENERATE_SPLIT_IN_CLIENT, LEAVE_AM_RUNNING, LOCAL_MODE, RECONNECT_APP_ID

Constructor Summary

Constructors

Constructor and Description
CartesianProduct()

Method Summary

Modifier and Type	Method and Description
static void	main(String[] args)
protected void	printUsage() Print usage instructions for this example
protected int	runJob(String[] args, org.apache.tez.dag.api.TezConfiguration tezConf, org.apache.tez.client.TezClient tezClient) Create and execute the actual DAG for the example
protected int	validateArgs(String[] otherArgs) Validate the arguments

Methods inherited from class org.apache.tez.examples.TezExampleBase

getAppId, isCountersLog, isDisableSplitGrouping, isGenerateSplitInClient, printExtraOptionsUsage, run, run, runDag

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hadoop.conf.Configurable

getConf, setConf

Constructor Detail

CartesianProduct

public CartesianProduct()

Method Detail

printUsage

protected void printUsage()

Description copied from class: TezExampleBase

Print usage instructions for this example

Specified by:

printUsage in class TezExampleBase

validateArgs

protected int validateArgs(String[] otherArgs)

Description copied from class: TezExampleBase

Validate the arguments

Specified by:

validateArgs in class TezExampleBase

Parameters:

otherArgs - arguments, if any

Returns:

Zero indicates success, non-zero indicates failure

runJob

protected int runJob(String[] args,
                     org.apache.tez.dag.api.TezConfiguration tezConf,
                     org.apache.tez.client.TezClient tezClient)
              throws Exception

Description copied from class: TezExampleBase

Create and execute the actual DAG for the example

Specified by:

runJob in class TezExampleBase

Parameters:

args - arguments for execution

tezConf - the tez configuration instance to be used while processing the DAG

tezClient - the tez client instance to use to run the DAG if any custom monitoring is required. Otherwise the utility method TezExampleBase.runDag(org.apache.tez.dag.api.DAG, boolean, org.slf4j.Logger) should be used

Returns:

Zero indicates success, non-zero indicates failure

Throws:

IOException

org.apache.tez.dag.api.TezException

Exception

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception