org.apache.beam.examples

Class DebuggingWordCount

java.lang.Object

org.apache.beam.examples.DebuggingWordCount

public class DebuggingWordCount
extends java.lang.Object

An example that verifies word counts in Shakespeare and includes Beam best practices.

This class, DebuggingWordCount, is the third in a series of four successively more detailed 'word count' examples. You may first want to take a look at MinimalWordCount and WordCount. After you've looked at this example, then see the WindowedWordCount pipeline, for introduction of additional concepts.

Basic concepts, also in the MinimalWordCount and WordCount examples: Reading text files; counting a PCollection; executing a Pipeline both locally and using a selected runner; defining DoFns.

New Concepts:

   1. Logging using SLF4J, even in a distributed environment
   2. Creating a custom metric (runners have varying levels of support)
   3. Testing your Pipeline via PAssert
 

To execute this pipeline locally, specify general pipeline configuration:

 --project=YOUR_PROJECT_ID
 

To change the runner, specify:

 --runner=YOUR_SELECTED_RUNNER
 

The input file defaults to a public data set containing the text of King Lear, by William Shakespeare. You can override it and choose your own input with --inputFile.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DebuggingWordCount.FilterTextFn A DoFn that filters for a specific key based upon a regular expression.
static interface	DebuggingWordCount.WordCountOptions Options supported by DebuggingWordCount.

Constructor Summary

Constructors

Constructor and Description
DebuggingWordCount()

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DebuggingWordCount

public DebuggingWordCount()

Method Detail

main

public static void main(java.lang.String[] args)