Package org.apache.beam.examples.common

Class ExampleUtils

  • public class ExampleUtils
extends java.lang.Object
    The utility class that sets up and tears down external resources, and cancels the streaming pipelines once the program terminates.

    It is used to run Beam examples.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String TOKENIZER_PATTERN
      \p{L} denotes the category of Unicode letters, so this pattern will match on everything that is not a letter.

    • Constructor Summary

      Constructors 
      Constructor Description
      ExampleUtils​(org.apache.beam.sdk.options.PipelineOptions options)
      Do resources and runner options setup.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void setup()
      Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.
      void setupBigQueryTable()
      Sets up the BigQuery table with the given schema.
      void setupPubsub()
      Sets up the Google Cloud Pub/Sub topic.
      void waitToFinish​(org.apache.beam.sdk.PipelineResult result)
      Waits for the pipeline to finish and cancels it before the program exists.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • TOKENIZER_PATTERN

        public static final java.lang.String TOKENIZER_PATTERN
        \p{L} denotes the category of Unicode letters, so this pattern will match on everything that is not a letter.

        It is used for tokenizing strings in the wordcount examples.

        See Also:
        Constant Field Values

    • Constructor Detail

      • ExampleUtils

        public ExampleUtils​(org.apache.beam.sdk.options.PipelineOptions options)
        Do resources and runner options setup.

    • Method Detail

      • setup

        public void setup()
           throws java.io.IOException
        Sets up external resources that are required by the example, such as Pub/Sub topics and BigQuery tables.
        Throws:
        java.io.IOException - if there is a problem setting up the resources

      • setupPubsub

        public void setupPubsub()
                 throws java.io.IOException
        Sets up the Google Cloud Pub/Sub topic.

        If the topic doesn't exist, a new topic with the given name will be created.

        Throws:
        java.io.IOException - if there is a problem setting up the Pub/Sub topic

      • setupBigQueryTable

        public void setupBigQueryTable()
                        throws java.io.IOException
        Sets up the BigQuery table with the given schema.

        If the table already exists, the schema has to match the given one. Otherwise, the example will throw a RuntimeException. If the table doesn't exist, a new table with the given schema will be created.

        Throws:
        java.io.IOException - if there is a problem setting up the BigQuery table

      • waitToFinish

        public void waitToFinish​(org.apache.beam.sdk.PipelineResult result)
        Waits for the pipeline to finish and cancels it before the program exists.