Package org.apache.beam.examples.cookbook

Class DistinctExample

java.lang.Object

org.apache.beam.examples.cookbook.DistinctExample

public class DistinctExample
extends java.lang.Object

This example uses as input text of King Lear, by William Shakespeare as plaintext files, and will remove any duplicate lines from this file. (The output does not preserve any input order).

Concepts: the Distinct transform, and how to wire transforms together. Demonstrates TextIO.Read/ Distinct/TextIO.Write.

To execute this pipeline locally, specify a local output file or output prefix on GCS: --output=[YOUR_LOCAL_FILE | gs://YOUR_OUTPUT_PREFIX]

To change the runner, specify:

 --runner=YOUR_SELECTED_RUNNER
 

See examples/java/README.md for instructions about how to configure different runners.

The input defaults to gs://apache-beam-samples/shakespeare/kinglear.txt and can be overridden with --input.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	DistinctExample.Options	Options supported by DistinctExample.

Constructor Summary

Constructors

Constructor	Description
DistinctExample()

Method Summary

Modifier and Type	Method	Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistinctExample

public DistinctExample()

Method Detail

main

public static void main(java.lang.String[] args)
                 throws java.lang.Exception

Throws:

java.lang.Exception