org.apache.beam.examples.cookbook

Class FilterExamples

java.lang.Object

org.apache.beam.examples.cookbook.FilterExamples

public class FilterExamples
extends java.lang.Object

This is an example that demonstrates several approaches to filtering, and use of the Mean transform. It shows how to dynamically set parameters by defining and using new pipeline options, and how to use a value derived by the pipeline.

Concepts: The Mean transform; Options configuration; using pipeline-derived data as a side input; approaches to filtering, selection, and projection.

The example reads public samples of weather data from BigQuery. It performs a projection on the data, finds the global mean of the temperature readings, filters on readings for a single given month, and then outputs only data (for that month) that has a mean temp smaller than the derived global mean.

Note: Before running this example, you must create a BigQuery dataset to contain your output table.

To execute this pipeline locally, specify the BigQuery table for the output:

 --output=YOUR_PROJECT_ID:DATASET_ID.TABLE_ID
 [--monthFilter=<month_number>]
 

where optional parameter --monthFilter is set to a number 1-12.

To change the runner, specify:

 --runner=YOUR_SELECTED_RUNNER
 

See examples/java/README.md for instructions about how to configure different runners.

The BigQuery input table defaults to clouddataflow-readonly:samples.weather_stations and can be overridden with --input.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	FilterExamples.Options Options supported by FilterExamples.

Constructor Summary

Constructors

Constructor and Description
FilterExamples()

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FilterExamples

public FilterExamples()

Method Detail

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception