org.apache.beam.fn.harness

Interface PTransformRunnerFactory<T>

All Known Implementing Classes:

CombineRunners.PrecombineFactory

public interface PTransformRunnerFactory<T>

A factory able to instantiate an appropriate handler for a given PTransform.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	PTransformRunnerFactory.Registrar A registrar which can return a mapping from RunnerApi.FunctionSpec#getUrn() to a factory capable of instantiating an appropriate handler.

Method Summary

Modifier and Type

Method and Description

T

createRunnerForPTransform(org.apache.beam.sdk.options.PipelineOptions pipelineOptions, BeamFnDataClient beamFnDataClient, BeamFnStateClient beamFnStateClient, java.lang.String pTransformId, org.apache.beam.model.pipeline.v1.RunnerApi.PTransform pTransform, java.util.function.Supplier<java.lang.String> processBundleInstructionId, java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.PCollection> pCollections, java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.Coder> coders, java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.WindowingStrategy> windowingStrategies, org.apache.beam.vendor.guava.v20_0.com.google.common.collect.ListMultimap<java.lang.String,org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>>> pCollectionIdsToConsumers, java.util.function.Consumer<org.apache.beam.sdk.fn.function.ThrowingRunnable> addStartFunction, java.util.function.Consumer<org.apache.beam.sdk.fn.function.ThrowingRunnable> addFinishFunction, BundleSplitListener splitListener) Creates and returns a handler for a given PTransform.

Method Detail

createRunnerForPTransform

T createRunnerForPTransform(org.apache.beam.sdk.options.PipelineOptions pipelineOptions,
                            BeamFnDataClient beamFnDataClient,
                            BeamFnStateClient beamFnStateClient,
                            java.lang.String pTransformId,
                            org.apache.beam.model.pipeline.v1.RunnerApi.PTransform pTransform,
                            java.util.function.Supplier<java.lang.String> processBundleInstructionId,
                            java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.PCollection> pCollections,
                            java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.Coder> coders,
                            java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.WindowingStrategy> windowingStrategies,
                            org.apache.beam.vendor.guava.v20_0.com.google.common.collect.ListMultimap<java.lang.String,org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>>> pCollectionIdsToConsumers,
                            java.util.function.Consumer<org.apache.beam.sdk.fn.function.ThrowingRunnable> addStartFunction,
                            java.util.function.Consumer<org.apache.beam.sdk.fn.function.ThrowingRunnable> addFinishFunction,
                            BundleSplitListener splitListener)
                     throws java.io.IOException

Creates and returns a handler for a given PTransform. Note that the handler must support processing multiple bundles. The handler will be discarded if an error is thrown during element processing, or during execution of start/finish.

Parameters:

pipelineOptions - Pipeline options

beamFnDataClient - A client for handling inbound and outbound data streams.

beamFnStateClient - A client for handling state requests.

pTransformId - The id of the PTransform.

pTransform - The PTransform definition.

processBundleInstructionId - A supplier containing the active process bundle instruction id.

pCollections - A mapping from PCollection id to PCollection definition.

coders - A mapping from coder id to coder definition.

windowingStrategies -

pCollectionIdsToConsumers - A mapping from PCollection id to a collection of consumers. Note that if this handler is a consumer, it should register itself within this multimap under the appropriate PCollection ids. Also note that all output consumers needed by this PTransform (based on the values of the RunnerApi.PTransform.getOutputsMap() will have already registered within this multimap.

addStartFunction - A consumer to register a start bundle handler with.

addFinishFunction - A consumer to register a finish bundle handler with.

splitListener - A listener to be invoked when the PTransform splits itself.

Throws:

java.io.IOException