org.apache.beam.examples.common

Class ExampleUtils

java.lang.Object

org.apache.beam.examples.common.ExampleUtils

public class ExampleUtils
extends java.lang.Object

The utility class that sets up and tears down external resources, and cancels the streaming pipelines once the program terminates.

It is used to run Beam examples.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	TOKENIZER_PATTERN \p{L} denotes the category of Unicode letters, so this pattern will match on everything that is not a letter.

Constructor Summary

Constructors

Constructor and Description
ExampleUtils(org.apache.beam.sdk.options.PipelineOptions options) Do resources and runner options setup.

Method Summary

Modifier and Type	Method and Description
void	setup() Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.
void	setupBigQueryTable() Sets up the BigQuery table with the given schema.
void	setupPubsub() Sets up the Google Cloud Pub/Sub topic.
void	waitToFinish(org.apache.beam.sdk.PipelineResult result) Waits for the pipeline to finish and cancels it before the program exists.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TOKENIZER_PATTERN

public static final java.lang.String TOKENIZER_PATTERN

\p{L} denotes the category of Unicode letters, so this pattern will match on everything that is not a letter.

It is used for tokenizing strings in the wordcount examples.

See Also:

Constant Field Values

Constructor Detail

ExampleUtils

public ExampleUtils(org.apache.beam.sdk.options.PipelineOptions options)

Do resources and runner options setup.

Method Detail

setup

public void setup()
           throws java.io.IOException

Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.

Throws:

java.io.IOException - if there is a problem setting up the resources

setupPubsub

public void setupPubsub()
                 throws java.io.IOException

Sets up the Google Cloud Pub/Sub topic.

If the topic doesn't exist, a new topic with the given name will be created.

Throws:

java.io.IOException - if there is a problem setting up the Pub/Sub topic

setupBigQueryTable

public void setupBigQueryTable()
                        throws java.io.IOException

Sets up the BigQuery table with the given schema.

If the table already exists, the schema has to match the given one. Otherwise, the example will throw a RuntimeException. If the table doesn't exist, a new table with the given schema will be created.

Throws:

java.io.IOException - if there is a problem setting up the BigQuery table

waitToFinish

public void waitToFinish(org.apache.beam.sdk.PipelineResult result)

Waits for the pipeline to finish and cancels it before the program exists.