org.apache.beam.runners.flink.translation.wrappers.streaming

Class WindowDoFnOperator<K,InputT,OutputT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<org.apache.beam.sdk.util.WindowedValue<OutputT>>

org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator<K,InputT,OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.flink.runtime.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.streaming.api.operators.Triggerable<java.lang.Object,org.apache.beam.runners.core.TimerInternals.TimerData>, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,org.apache.beam.sdk.transforms.join.RawUnionValue,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.util.Disposable

public class WindowDoFnOperator<K,InputT,OutputT>
extends DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

Flink operator for executing window DoFns.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

DoFnOperator.BufferedOutputManager<OutputT>, DoFnOperator.FlinkStepContext, DoFnOperator.MultiOutputOutputManagerFactory<OutputT>

Nested classes/interfaces inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

org.apache.flink.streaming.api.operators.AbstractStreamOperator.CountingOutput

Field Summary

Fields inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

additionalOutputTags, bufferingDoFnRunner, currentInputWatermark, currentOutputWatermark, currentSideInputWatermark, doFn, doFnRunner, keyCoder, keyedStateInternals, mainOutputTag, outputManager, outputManagerFactory, pushbackDoFnRunner, serializedOptions, sideInputHandler, sideInputReader, sideInputs, sideInputTagMapping, stepName, timerInternals, timerService, windowingStrategy

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, LOG, metrics, output, timeServiceManager

Constructor Summary

Constructors

Constructor and Description

WindowDoFnOperator(org.apache.beam.runners.core.SystemReduceFn<K,InputT,?,OutputT,org.apache.beam.sdk.transforms.windowing.BoundedWindow> systemReduceFn, java.lang.String stepName, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>> windowedInputCoder, org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,OutputT>> mainOutputTag, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags, org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.OutputManagerFactory<org.apache.beam.sdk.values.KV<K,OutputT>> outputManagerFactory, org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy, java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputTagMapping, java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs, org.apache.beam.sdk.options.PipelineOptions options, org.apache.beam.sdk.coders.Coder<K> keyCoder, org.apache.flink.api.java.functions.KeySelector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,?> keySelector)

Method Summary

Modifier and Type	Method and Description
protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>	createWrappingDoFnRunner(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> wrappedRunner)
void	fireTimer(org.apache.flink.streaming.api.operators.InternalTimer<?,org.apache.beam.runners.core.TimerInternals.TimerData> timer)
protected org.apache.beam.sdk.transforms.DoFn<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>	getDoFn()

Methods inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

addSideInputValue, close, dispose, getLockToAcquireForStateAccessDuringBundles, getPushbackWatermarkHold, initializeState, invokeFinishBundle, notifyCheckpointComplete, onEventTime, onProcessingTime, open, processElement, processElement1, processElement2, processWatermark, processWatermark1, processWatermark2, setBundleFinishedCallback, setPushedBackWatermark, setup, snapshotState

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getUserCodeClassloader, initializeState, numEventTimeTimers, numProcessingTimeTimers, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

processLatencyMarker

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

getChainingStrategy, getMetricGroup, getOperatorID, initializeState, setChainingStrategy, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Constructor Detail

WindowDoFnOperator

public WindowDoFnOperator(org.apache.beam.runners.core.SystemReduceFn<K,InputT,?,OutputT,org.apache.beam.sdk.transforms.windowing.BoundedWindow> systemReduceFn,
                          java.lang.String stepName,
                          org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>> windowedInputCoder,
                          org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,OutputT>> mainOutputTag,
                          java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags,
                          org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.OutputManagerFactory<org.apache.beam.sdk.values.KV<K,OutputT>> outputManagerFactory,
                          org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy,
                          java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputTagMapping,
                          java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs,
                          org.apache.beam.sdk.options.PipelineOptions options,
                          org.apache.beam.sdk.coders.Coder<K> keyCoder,
                          org.apache.flink.api.java.functions.KeySelector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,?> keySelector)

Method Detail

createWrappingDoFnRunner

protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> createWrappingDoFnRunner(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> wrappedRunner)

Overrides:

createWrappingDoFnRunner in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

getDoFn

protected org.apache.beam.sdk.transforms.DoFn<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> getDoFn()

Overrides:

getDoFn in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

fireTimer

public void fireTimer(org.apache.flink.streaming.api.operators.InternalTimer<?,org.apache.beam.runners.core.TimerInternals.TimerData> timer)

Overrides:

fireTimer in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>