org.apache.beam.runners.flink.translation.functions

Class FlinkExecutableStageFunction<InputT>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.beam.runners.flink.translation.functions.FlinkExecutableStageFunction<InputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.GroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>, org.apache.flink.api.common.functions.MapPartitionFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>, org.apache.flink.api.common.functions.RichFunction

public class FlinkExecutableStageFunction<InputT>
extends org.apache.flink.api.common.functions.AbstractRichFunction
implements org.apache.flink.api.common.functions.MapPartitionFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>, org.apache.flink.api.common.functions.GroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>

Flink operator that passes its input DataSet through an SDK-executed ExecutableStage.

The output of this operation is a multiplexed DataSet whose elements are tagged with a union coder. The coder's tags are determined by the output coder map. The resulting data set should be further processed by a FlinkExecutableStagePruningFunction.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

FlinkExecutableStageFunction(java.lang.String stepName, org.apache.beam.sdk.options.PipelineOptions pipelineOptions, org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload, org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo, java.util.Map<java.lang.String,java.lang.Integer> outputMap, FlinkExecutableStageContextFactory contextFactory, org.apache.beam.sdk.coders.Coder windowCoder, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InputT>> inputCoder)

Method Summary

Modifier and Type	Method and Description
void	close()
void	mapPartition(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<InputT>> iterable, org.apache.flink.util.Collector<org.apache.beam.sdk.transforms.join.RawUnionValue> collector) For non-stateful processing via a simple MapPartitionFunction.
void	open(org.apache.flink.configuration.Configuration parameters)
void	reduce(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<InputT>> iterable, org.apache.flink.util.Collector<org.apache.beam.sdk.transforms.join.RawUnionValue> collector) For stateful and timer processing via a GroupReduceFunction.

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkExecutableStageFunction

public FlinkExecutableStageFunction(java.lang.String stepName,
                                    org.apache.beam.sdk.options.PipelineOptions pipelineOptions,
                                    org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload,
                                    org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo,
                                    java.util.Map<java.lang.String,java.lang.Integer> outputMap,
                                    FlinkExecutableStageContextFactory contextFactory,
                                    org.apache.beam.sdk.coders.Coder windowCoder,
                                    org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InputT>> inputCoder)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

mapPartition

public void mapPartition(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<InputT>> iterable,
                         org.apache.flink.util.Collector<org.apache.beam.sdk.transforms.join.RawUnionValue> collector)
                  throws java.lang.Exception

For non-stateful processing via a simple MapPartitionFunction.

Specified by:

mapPartition in interface org.apache.flink.api.common.functions.MapPartitionFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>

Throws:

java.lang.Exception

reduce

public void reduce(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<InputT>> iterable,
                   org.apache.flink.util.Collector<org.apache.beam.sdk.transforms.join.RawUnionValue> collector)
            throws java.lang.Exception

For stateful and timer processing via a GroupReduceFunction.

Specified by:

reduce in interface org.apache.flink.api.common.functions.GroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<InputT>,org.apache.beam.sdk.transforms.join.RawUnionValue>

Throws:

java.lang.Exception

close

public void close()
           throws java.lang.Exception

Specified by:

close in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

close in class org.apache.flink.api.common.functions.AbstractRichFunction

Throws:

java.lang.Exception