org.apache.beam.runners.flink.translation.wrappers.streaming

Class WindowDoFnOperator<K,InputT,OutputT>

java.lang.Object

org.apache.flink.streaming.api.operators.AbstractStreamOperator<OutputT>

org.apache.beam.runners.flink.translation.wrappers.streaming.AbstractStreamOperatorCompat<org.apache.beam.sdk.util.WindowedValue<OutputT>>

org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

org.apache.beam.runners.flink.translation.wrappers.streaming.WindowDoFnOperator<K,InputT,OutputT>

All Implemented Interfaces:

java.io.Serializable, org.apache.flink.runtime.state.CheckpointListener, org.apache.flink.streaming.api.operators.KeyContext, org.apache.flink.streaming.api.operators.OneInputStreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.streaming.api.operators.SetupableStreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.streaming.api.operators.StreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.streaming.api.operators.StreamOperatorStateHandler.CheckpointedStreamOperator, org.apache.flink.streaming.api.operators.Triggerable<java.nio.ByteBuffer,org.apache.beam.runners.core.TimerInternals.TimerData>, org.apache.flink.streaming.api.operators.TwoInputStreamOperator<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,org.apache.beam.sdk.transforms.join.RawUnionValue,org.apache.beam.sdk.util.WindowedValue<org.apache.beam.sdk.values.KV<K,OutputT>>>, org.apache.flink.util.Disposable

public class WindowDoFnOperator<K,InputT,OutputT>
extends DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

Flink operator for executing window DoFns.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

DoFnOperator.BufferedOutputManager<OutputT>, DoFnOperator.FlinkStepContext, DoFnOperator.MultiOutputOutputManagerFactory<OutputT>

Field Summary

Fields inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

additionalOutputTags, bufferingDoFnRunner, doFn, doFnRunner, keyCoder, keyedStateInternals, mainOutputTag, outputManager, outputManagerFactory, pushbackDoFnRunner, serializedOptions, sideInputHandler, sideInputReader, sideInputs, sideInputTagMapping, stepName, timerInternals, timerService, windowingStrategy

Fields inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

chainingStrategy, config, latencyStats, metrics, output, processingTimeService

Constructor Summary

Constructors

Constructor and Description

WindowDoFnOperator(org.apache.beam.runners.core.SystemReduceFn<K,InputT,?,OutputT,org.apache.beam.sdk.transforms.windowing.BoundedWindow> systemReduceFn, java.lang.String stepName, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>> windowedInputCoder, org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,OutputT>> mainOutputTag, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags, org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.OutputManagerFactory<org.apache.beam.sdk.values.KV<K,OutputT>> outputManagerFactory, org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy, java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputTagMapping, java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs, org.apache.beam.sdk.options.PipelineOptions options, org.apache.beam.sdk.coders.Coder<K> keyCoder, org.apache.flink.api.java.functions.KeySelector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,?> keySelector)

Method Summary

Modifier and Type	Method and Description
protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>	createWrappingDoFnRunner(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> wrappedRunner, org.apache.beam.runners.core.StepContext stepContext)
protected void	fireTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)
protected org.apache.beam.sdk.transforms.DoFn<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>	getDoFn()

Methods inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator

addSideInputValue, applyInputWatermarkHold, applyOutputWatermarkHold, fireTimerInternal, getBundleFinalizer, getCurrentOutputWatermark, getEffectiveInputWatermark, getLockToAcquireForStateAccessDuringBundles, initializeState, invokeFinishBundle, notifyCheckpointComplete, onEventTime, onProcessingTime, open, prepareSnapshotPreBarrier, processElement, processElement1, processElement2, processWatermark, processWatermark1, processWatermark2, setBundleFinishedCallback, setPreBundleCallback, setup, snapshotState

Methods inherited from class org.apache.beam.runners.flink.translation.wrappers.streaming.AbstractStreamOperatorCompat

close, dispose, getTimeServiceManagerCompat

Methods inherited from class org.apache.flink.streaming.api.operators.AbstractStreamOperator

getChainingStrategy, getContainingTask, getCurrentKey, getExecutionConfig, getInternalTimerService, getKeyedStateBackend, getKeyedStateStore, getMetricGroup, getOperatorConfig, getOperatorID, getOperatorName, getOperatorStateBackend, getOrCreateKeyedState, getPartitionedState, getPartitionedState, getProcessingTimeService, getRuntimeContext, getTimeServiceManager, getUserCodeClassloader, initializeState, isUsingCustomRawKeyedState, notifyCheckpointAborted, numEventTimeTimers, numProcessingTimeTimers, processLatencyMarker, processLatencyMarker1, processLatencyMarker2, reportOrForwardLatencyMarker, setChainingStrategy, setCurrentKey, setKeyContextElement1, setKeyContextElement2, setProcessingTimeService, snapshotState

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.flink.streaming.api.operators.OneInputStreamOperator

processLatencyMarker

Methods inherited from interface org.apache.flink.streaming.api.operators.TwoInputStreamOperator

processLatencyMarker1, processLatencyMarker2

Methods inherited from interface org.apache.flink.streaming.api.operators.StreamOperator

close, dispose, getMetricGroup, getOperatorID, initializeState, setKeyContextElement1, setKeyContextElement2, snapshotState

Methods inherited from interface org.apache.flink.runtime.state.CheckpointListener

notifyCheckpointAborted

Methods inherited from interface org.apache.flink.streaming.api.operators.KeyContext

getCurrentKey, setCurrentKey

Constructor Detail

WindowDoFnOperator

public WindowDoFnOperator(org.apache.beam.runners.core.SystemReduceFn<K,InputT,?,OutputT,org.apache.beam.sdk.transforms.windowing.BoundedWindow> systemReduceFn,
                          java.lang.String stepName,
                          org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>> windowedInputCoder,
                          org.apache.beam.sdk.values.TupleTag<org.apache.beam.sdk.values.KV<K,OutputT>> mainOutputTag,
                          java.util.List<org.apache.beam.sdk.values.TupleTag<?>> additionalOutputTags,
                          org.apache.beam.runners.flink.translation.wrappers.streaming.DoFnOperator.OutputManagerFactory<org.apache.beam.sdk.values.KV<K,OutputT>> outputManagerFactory,
                          org.apache.beam.sdk.values.WindowingStrategy<?,?> windowingStrategy,
                          java.util.Map<java.lang.Integer,org.apache.beam.sdk.values.PCollectionView<?>> sideInputTagMapping,
                          java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs,
                          org.apache.beam.sdk.options.PipelineOptions options,
                          org.apache.beam.sdk.coders.Coder<K> keyCoder,
                          org.apache.flink.api.java.functions.KeySelector<org.apache.beam.sdk.util.WindowedValue<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>>,?> keySelector)

Method Detail

createWrappingDoFnRunner

protected org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> createWrappingDoFnRunner(org.apache.beam.runners.core.DoFnRunner<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> wrappedRunner,
                                                                                                                                                                          org.apache.beam.runners.core.StepContext stepContext)

Overrides:

createWrappingDoFnRunner in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

getDoFn

protected org.apache.beam.sdk.transforms.DoFn<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>> getDoFn()

Overrides:

getDoFn in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>

fireTimer

protected void fireTimer(org.apache.beam.runners.core.TimerInternals.TimerData timer)

Overrides:

fireTimer in class DoFnOperator<org.apache.beam.runners.core.KeyedWorkItem<K,InputT>,org.apache.beam.sdk.values.KV<K,OutputT>>