org.apache.beam.sdk.extensions.python.transforms

Class PythonMap<InputT,OutputT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<? extends InputT>,org.apache.beam.sdk.values.PCollection<OutputT>>

org.apache.beam.sdk.extensions.python.transforms.PythonMap<InputT,OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public class PythonMap<InputT,OutputT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<? extends InputT>,org.apache.beam.sdk.values.PCollection<OutputT>>

Wrapper for invoking external Python Map transforms..

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<OutputT>	expand(org.apache.beam.sdk.values.PCollection<? extends InputT> input)
static <InputT,OutputT> PythonMap<InputT,OutputT>	viaFlatMapFn(java.lang.String pythonFunction, org.apache.beam.sdk.coders.Coder<?> outputCoder)
static <InputT,OutputT> PythonMap<InputT,OutputT>	viaMapFn(java.lang.String pythonFunction, org.apache.beam.sdk.coders.Coder<?> outputCoder)
PythonMap<InputT,OutputT>	withExpansionService(java.lang.String expansionService)
PythonMap<InputT,OutputT>	withExtraPackages(java.util.List<java.lang.String> extraPackages) Specifies any extra packages required by the Python function.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

viaMapFn

public static <InputT,OutputT> PythonMap<InputT,OutputT> viaMapFn(java.lang.String pythonFunction,
                                                                  org.apache.beam.sdk.coders.Coder<?> outputCoder)

viaFlatMapFn

public static <InputT,OutputT> PythonMap<InputT,OutputT> viaFlatMapFn(java.lang.String pythonFunction,
                                                                      org.apache.beam.sdk.coders.Coder<?> outputCoder)

withExpansionService

public PythonMap<InputT,OutputT> withExpansionService(java.lang.String expansionService)

withExtraPackages

public PythonMap<InputT,OutputT> withExtraPackages(java.util.List<java.lang.String> extraPackages)

Specifies any extra packages required by the Python function.

This should only be specified when using the default expansion service, i.e. when not using withExpansionService(String) to provide an expansion service.

The package can either be a PyPi package or the path to a locally available Python package.

Parameters:

extraPackages - a list of PyPi packages. May include the version.

Returns:

A PythonMap with extra packages.

expand

public org.apache.beam.sdk.values.PCollection<OutputT> expand(org.apache.beam.sdk.values.PCollection<? extends InputT> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<? extends InputT>,org.apache.beam.sdk.values.PCollection<OutputT>>