org.apache.beam.runners.flink.translation.functions

Class FlinkMergingNonShuffleReduceFunction<K,InputT,AccumT,OutputT,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow>

java.lang.Object

org.apache.flink.api.common.functions.AbstractRichFunction

org.apache.flink.api.common.functions.RichGroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>

org.apache.beam.runners.flink.translation.functions.FlinkMergingNonShuffleReduceFunction<K,InputT,AccumT,OutputT,W>

All Implemented Interfaces:

java.io.Serializable, org.apache.flink.api.common.functions.Function, org.apache.flink.api.common.functions.GroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.api.common.functions.RichFunction

public class FlinkMergingNonShuffleReduceFunction<K,InputT,AccumT,OutputT,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow>
extends org.apache.flink.api.common.functions.RichGroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>

Special version of FlinkReduceFunction that supports merging windows.

This is different from the pair of function for the non-merging windows case in that we cannot do combining before the shuffle because elements would not yet be in their correct windows for side-input access.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
FlinkMergingNonShuffleReduceFunction(org.apache.beam.sdk.transforms.CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> combineFn, org.apache.beam.sdk.values.WindowingStrategy<java.lang.Object,W> windowingStrategy, java.util.Map<org.apache.beam.sdk.values.PCollectionView<?>,org.apache.beam.sdk.values.WindowingStrategy<?,?>> sideInputs, org.apache.beam.sdk.options.PipelineOptions pipelineOptions)

Method Summary

Modifier and Type	Method and Description
void	open(org.apache.flink.configuration.Configuration parameters)
void	reduce(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>> elements, org.apache.flink.util.Collector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>> out)

Methods inherited from class org.apache.flink.api.common.functions.AbstractRichFunction

close, getIterationRuntimeContext, getRuntimeContext, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FlinkMergingNonShuffleReduceFunction

public FlinkMergingNonShuffleReduceFunction(org.apache.beam.sdk.transforms.CombineFnBase.GlobalCombineFn<InputT,AccumT,OutputT> combineFn,
                                            org.apache.beam.sdk.values.WindowingStrategy<java.lang.Object,W> windowingStrategy,
                                            java.util.Map<org.apache.beam.sdk.values.PCollectionView<?>,org.apache.beam.sdk.values.WindowingStrategy<?,?>> sideInputs,
                                            org.apache.beam.sdk.options.PipelineOptions pipelineOptions)

Method Detail

open

public void open(org.apache.flink.configuration.Configuration parameters)

Specified by:

open in interface org.apache.flink.api.common.functions.RichFunction

Overrides:

open in class org.apache.flink.api.common.functions.AbstractRichFunction

reduce

public void reduce(java.lang.Iterable<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>> elements,
                   org.apache.flink.util.Collector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>> out)
            throws java.lang.Exception

Specified by:

reduce in interface org.apache.flink.api.common.functions.GroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>

Specified by:

reduce in class org.apache.flink.api.common.functions.RichGroupReduceFunction<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>

Throws:

java.lang.Exception