org.apache.beam.examples.multilanguage

Class PythonDataframeWordCount

java.lang.Object

org.apache.beam.examples.multilanguage.PythonDataframeWordCount

public class PythonDataframeWordCount
extends java.lang.Object

An example that counts words in Shakespeare and utilizes a Python external transform.

This class, PythonDataframeWordCount, uses Python DataframeTransform to count words from the input text file.

The example command below shows how to run this pipeline on Dataflow runner:

 ./gradlew :examples:multi-language:pythonDataframeWordCount --args=" \
 --runner=DataflowRunner \
 --output=gs://{$OUTPUT_BUCKET}/count \
 --sdkHarnessContainerImageOverrides=.*python.*,gcr.io/apache-beam-testing/beam-sdk/beam_python{$PYTHON_VERSION}_sdk:latest"
 

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	PythonDataframeWordCount.FormatAsTextFn A SimpleFunction that converts a counted row into a printable string.
static interface	PythonDataframeWordCount.WordCountOptions Options supported by PythonDataframeWordCount.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	TOKENIZER_PATTERN

Constructor Summary

Constructors

Constructor and Description
PythonDataframeWordCount()

Method Summary

Modifier and Type	Method and Description
static void	main(java.lang.String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TOKENIZER_PATTERN

public static final java.lang.String TOKENIZER_PATTERN

See Also:

Constant Field Values

Constructor Detail

PythonDataframeWordCount

public PythonDataframeWordCount()

Method Detail

main

public static void main(java.lang.String[] args)