org.apache.beam.runners.samza.runtime

Class PortableDoFnOp<InT,FnOutT,OutT>

java.lang.Object

org.apache.beam.runners.samza.runtime.PortableDoFnOp<InT,FnOutT,OutT>

All Implemented Interfaces:

java.io.Serializable, Op<InT,OutT,java.lang.Void>

public class PortableDoFnOp<InT,FnOutT,OutT>
extends java.lang.Object
implements Op<InT,OutT,java.lang.Void>

Samza operator for DoFn.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description

PortableDoFnOp(org.apache.beam.sdk.values.TupleTag<FnOutT> mainOutputTag, org.apache.beam.sdk.transforms.DoFn<InT,FnOutT> doFn, org.apache.beam.sdk.coders.Coder<?> keyCoder, org.apache.beam.sdk.coders.Coder<InT> inputCoder, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InT>> windowedValueCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders, java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> sideOutputTags, org.apache.beam.sdk.values.WindowingStrategy windowingStrategy, java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> idToViewMap, OutputManagerFactory<OutT> outputManagerFactory, java.lang.String transformFullName, java.lang.String transformId, org.apache.beam.sdk.values.PCollection.IsBounded isBounded, boolean isPortable, org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload, org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo, java.util.Map<java.lang.String,org.apache.beam.sdk.values.TupleTag<?>> idToTupleTagMap, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, java.util.Map<?,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping, java.util.Map<java.lang.String,java.lang.String> stateIdToStoreMapping)

Method Summary

Modifier and Type	Method and Description
void	close()
void	open(org.apache.samza.config.Config config, org.apache.samza.context.Context context, org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> timerRegistry, OpEmitter<OutT> emitter) A hook that allows initialization for any non-serializable operator state, such as getting stores.
void	processElement(org.apache.beam.sdk.util.WindowedValue<InT> inputElement, OpEmitter<OutT> emitter)
void	processSideInput(java.lang.String id, org.apache.beam.sdk.util.WindowedValue<? extends java.lang.Iterable<?>> elements, OpEmitter<OutT> emitter)
void	processSideInputWatermark(org.joda.time.Instant watermark, OpEmitter<OutT> emitter)
void	processTimer(KeyedTimerData<java.lang.Void> keyedTimerData, OpEmitter<OutT> emitter)
void	processWatermark(org.joda.time.Instant watermark, OpEmitter<OutT> emitter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PortableDoFnOp

public PortableDoFnOp(org.apache.beam.sdk.values.TupleTag<FnOutT> mainOutputTag,
                      org.apache.beam.sdk.transforms.DoFn<InT,FnOutT> doFn,
                      org.apache.beam.sdk.coders.Coder<?> keyCoder,
                      org.apache.beam.sdk.coders.Coder<InT> inputCoder,
                      org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InT>> windowedValueCoder,
                      java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders,
                      java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs,
                      java.util.List<org.apache.beam.sdk.values.TupleTag<?>> sideOutputTags,
                      org.apache.beam.sdk.values.WindowingStrategy windowingStrategy,
                      java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> idToViewMap,
                      OutputManagerFactory<OutT> outputManagerFactory,
                      java.lang.String transformFullName,
                      java.lang.String transformId,
                      org.apache.beam.sdk.values.PCollection.IsBounded isBounded,
                      boolean isPortable,
                      org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload,
                      org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo,
                      java.util.Map<java.lang.String,org.apache.beam.sdk.values.TupleTag<?>> idToTupleTagMap,
                      org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
                      java.util.Map<?,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping,
                      java.util.Map<java.lang.String,java.lang.String> stateIdToStoreMapping)

Method Detail

open

public void open(org.apache.samza.config.Config config,
                 org.apache.samza.context.Context context,
                 org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> timerRegistry,
                 OpEmitter<OutT> emitter)

Description copied from interface: Op

A hook that allows initialization for any non-serializable operator state, such as getting stores.

While an emitter is supplied to this function it is not usable except in the methods Op.processElement(WindowedValue, OpEmitter), Op.processWatermark(Instant, OpEmitter), and Op.processSideInput(String, WindowedValue, OpEmitter).

Specified by:

open in interface Op<InT,OutT,java.lang.Void>

processElement

public void processElement(org.apache.beam.sdk.util.WindowedValue<InT> inputElement,
                           OpEmitter<OutT> emitter)

Specified by:

processElement in interface Op<InT,OutT,java.lang.Void>

processWatermark

public void processWatermark(org.joda.time.Instant watermark,
                             OpEmitter<OutT> emitter)

Specified by:

processWatermark in interface Op<InT,OutT,java.lang.Void>

processSideInput

public void processSideInput(java.lang.String id,
                             org.apache.beam.sdk.util.WindowedValue<? extends java.lang.Iterable<?>> elements,
                             OpEmitter<OutT> emitter)

Specified by:

processSideInput in interface Op<InT,OutT,java.lang.Void>

processSideInputWatermark

public void processSideInputWatermark(org.joda.time.Instant watermark,
                                      OpEmitter<OutT> emitter)

Specified by:

processSideInputWatermark in interface Op<InT,OutT,java.lang.Void>

processTimer

public void processTimer(KeyedTimerData<java.lang.Void> keyedTimerData,
                         OpEmitter<OutT> emitter)

Specified by:

processTimer in interface Op<InT,OutT,java.lang.Void>

close

public void close()

Specified by:

close in interface Op<InT,OutT,java.lang.Void>