org.apache.beam.runners.samza.runtime

Class DoFnOp<InT,FnOutT,OutT>

java.lang.Object

org.apache.beam.runners.samza.runtime.DoFnOp<InT,FnOutT,OutT>

All Implemented Interfaces:

java.io.Serializable, Op<InT,OutT,java.lang.Void>

public class DoFnOp<InT,FnOutT,OutT>
extends java.lang.Object
implements Op<InT,OutT,java.lang.Void>

Samza operator for DoFn.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DoFnOp.MultiOutputManagerFactory Factory class to create an DoFnRunners.OutputManager that emits values to the main output as well as the side outputs via union type RawUnionValue.
static class	DoFnOp.SingleOutputManagerFactory<OutT> Factory class to create an DoFnRunners.OutputManager that emits values to the main output only, which is a single PCollection.

Constructor Summary

Constructors

Constructor and Description

DoFnOp(org.apache.beam.sdk.values.TupleTag<FnOutT> mainOutputTag, org.apache.beam.sdk.transforms.DoFn<InT,FnOutT> doFn, org.apache.beam.sdk.coders.Coder<?> keyCoder, org.apache.beam.sdk.coders.Coder<InT> inputCoder, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InT>> windowedValueCoder, java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders, java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs, java.util.List<org.apache.beam.sdk.values.TupleTag<?>> sideOutputTags, org.apache.beam.sdk.values.WindowingStrategy windowingStrategy, java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> idToViewMap, OutputManagerFactory<OutT> outputManagerFactory, java.lang.String transformFullName, java.lang.String transformId, org.apache.beam.sdk.values.PCollection.IsBounded isBounded, boolean isPortable, org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload, org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo, java.util.Map<java.lang.String,org.apache.beam.sdk.values.TupleTag<?>> idToTupleTagMap, org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation, java.util.Map<?,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping, java.util.Map<java.lang.String,java.lang.String> stateIdToStoreMapping)

Method Summary

Modifier and Type	Method and Description
void	close()
void	open(org.apache.samza.config.Config config, org.apache.samza.context.Context context, org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> timerRegistry, OpEmitter<OutT> emitter) A hook that allows initialization for any non-serializable operator state, such as getting stores.
void	processElement(org.apache.beam.sdk.util.WindowedValue<InT> inputElement, OpEmitter<OutT> emitter)
void	processSideInput(java.lang.String id, org.apache.beam.sdk.util.WindowedValue<? extends java.lang.Iterable<?>> elements, OpEmitter<OutT> emitter)
void	processSideInputWatermark(org.joda.time.Instant watermark, OpEmitter<OutT> emitter)
void	processTimer(KeyedTimerData<java.lang.Void> keyedTimerData, OpEmitter<OutT> emitter)
void	processWatermark(org.joda.time.Instant watermark, OpEmitter<OutT> emitter)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DoFnOp

public DoFnOp(org.apache.beam.sdk.values.TupleTag<FnOutT> mainOutputTag,
              org.apache.beam.sdk.transforms.DoFn<InT,FnOutT> doFn,
              org.apache.beam.sdk.coders.Coder<?> keyCoder,
              org.apache.beam.sdk.coders.Coder<InT> inputCoder,
              org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<InT>> windowedValueCoder,
              java.util.Map<org.apache.beam.sdk.values.TupleTag<?>,org.apache.beam.sdk.coders.Coder<?>> outputCoders,
              java.util.Collection<org.apache.beam.sdk.values.PCollectionView<?>> sideInputs,
              java.util.List<org.apache.beam.sdk.values.TupleTag<?>> sideOutputTags,
              org.apache.beam.sdk.values.WindowingStrategy windowingStrategy,
              java.util.Map<java.lang.String,org.apache.beam.sdk.values.PCollectionView<?>> idToViewMap,
              OutputManagerFactory<OutT> outputManagerFactory,
              java.lang.String transformFullName,
              java.lang.String transformId,
              org.apache.beam.sdk.values.PCollection.IsBounded isBounded,
              boolean isPortable,
              org.apache.beam.model.pipeline.v1.RunnerApi.ExecutableStagePayload stagePayload,
              org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo,
              java.util.Map<java.lang.String,org.apache.beam.sdk.values.TupleTag<?>> idToTupleTagMap,
              org.apache.beam.sdk.transforms.DoFnSchemaInformation doFnSchemaInformation,
              java.util.Map<?,org.apache.beam.sdk.values.PCollectionView<?>> sideInputMapping,
              java.util.Map<java.lang.String,java.lang.String> stateIdToStoreMapping)

Method Detail

open

public void open(org.apache.samza.config.Config config,
                 org.apache.samza.context.Context context,
                 org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> timerRegistry,
                 OpEmitter<OutT> emitter)

Description copied from interface: Op

A hook that allows initialization for any non-serializable operator state, such as getting stores.

While an emitter is supplied to this function it is not usable except in the methods Op.processElement(WindowedValue, OpEmitter), Op.processWatermark(Instant, OpEmitter), and Op.processSideInput(String, WindowedValue, OpEmitter).

Specified by:

open in interface Op<InT,OutT,java.lang.Void>

processElement

public void processElement(org.apache.beam.sdk.util.WindowedValue<InT> inputElement,
                           OpEmitter<OutT> emitter)

Specified by:

processElement in interface Op<InT,OutT,java.lang.Void>

processWatermark

public void processWatermark(org.joda.time.Instant watermark,
                             OpEmitter<OutT> emitter)

Specified by:

processWatermark in interface Op<InT,OutT,java.lang.Void>

processSideInput

public void processSideInput(java.lang.String id,
                             org.apache.beam.sdk.util.WindowedValue<? extends java.lang.Iterable<?>> elements,
                             OpEmitter<OutT> emitter)

Specified by:

processSideInput in interface Op<InT,OutT,java.lang.Void>

processSideInputWatermark

public void processSideInputWatermark(org.joda.time.Instant watermark,
                                      OpEmitter<OutT> emitter)

Specified by:

processSideInputWatermark in interface Op<InT,OutT,java.lang.Void>

processTimer

public void processTimer(KeyedTimerData<java.lang.Void> keyedTimerData,
                         OpEmitter<OutT> emitter)

Specified by:

processTimer in interface Op<InT,OutT,java.lang.Void>

close

public void close()

Specified by:

close in interface Op<InT,OutT,java.lang.Void>