org.apache.beam.runners.flink.translation.functions

Class FlinkDoFnFunction<InputT,OutputT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.beam.runners.flink.translation.functions.FlinkDoFnFunction<InputT,OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.flink.api.common.functions.FlatMapFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.transforms.join.RawUnionValue>>, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.RichFunction

public class FlinkDoFnFunction<InputT,OutputT>
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.api.common.functions.FlatMapFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.transforms.join.RawUnionValue>>

Encapsulates a DoFn inside a Flink RichMapPartitionFunction.

We get a mapping from TupleTag to output index and must tag all outputs with the output number. Afterwards a filter will filter out those elements that are not to be in a specific output.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

FlinkDoFnFunction(org.apache.beam.sdk.transforms.DoFn<InputT,OutputT> doFn, java.lang.String stepName, org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy, java.util.Map<org.apache.beam.sdk.values.PCollectionView<?>,org.apache.beam.sdk.values.WindowingStrategy<?,?>> sideInputs, org.apache.beam.sdk.options.PipelineOptions options, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,java.lang.Integer> outputMap, org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag, org.apache.beam.sdk.coders.Coder<InputT> inputCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoderMap, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)

Method Summary

Modifier and Type	Method and Description
void	close()
void	flatMap(org.apache.beam.sdk.util.WindowedValue<InputT> value, org.apache.flink.util.Collector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.transforms.join.RawUnionValue>> out)
void	open(org.apache.flink.configuration.Configuration parameters)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkDoFnFunction

public FlinkDoFnFunction(org.apache.beam.sdk.transforms.DoFn<InputT,OutputT> doFn,
                         java.lang.String stepName,
                         org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy,
                         java.util.Map<org.apache.beam.sdk.values.PCollectionView<?>,org.apache.beam.sdk.values.WindowingStrategy<?,?>> sideInputs,
                         org.apache.beam.sdk.options.PipelineOptions options,
                         java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,java.lang.Integer> outputMap,
                         org.apache.beam.sdk.values.TupleTag<OutputT> mainOutputTag,
                         org.apache.beam.sdk.coders.Coder<InputT> inputCoder,
                         java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoderMap,
                         org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
                         java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping)

Method Detail

flatMap

public void flatMap(org.apache.beam.sdk.util.WindowedValue<InputT> value,
                    org.apache.flink.util.Collector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.transforms.join.RawUnionValue>> out)

Specified by:

flatMap in interface org.apache.flink.api.common.functions.FlatMapFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.transforms.join.RawUnionValue>>

open

public void open(org.apache.flink.configuration.Configuration parameters)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

close

public void close()
           throws java.lang.Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

java.lang.Exception