org.apache.beam.fn.harness

Interface PTransformRunnerFactory.Context

Enclosing interface:

PTransformRunnerFactory<T>

public static interface PTransformRunnerFactory.Context

A context used to instantiate and support the handler necessary to execute the PTransform.

Method Summary

Modifier and Type	Method and Description
void	addBundleProgressReporter(BundleProgressReporter bundleProgressReporter) Register a callback whenever progress is being requested.
void	addFinishBundleFunction(org.apache.beam.sdk.function.ThrowingRunnable finishBundleFunction) Register any DoFn.FinishBundle methods.
<T> void	addIncomingDataEndpoint(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, org.apache.beam.sdk.coders.Coder<T> coder, org.apache.beam.sdk.fn.data.FnDataReceiver<T> receiver)
<T> void	addIncomingTimerEndpoint(java.lang.String timerFamilyId, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.construction.Timer<T>> coder, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.construction.Timer<T>> receiver)
<T> org.apache.beam.sdk.fn.data.FnDataReceiver<T>	addOutgoingDataEndpoint(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, org.apache.beam.sdk.coders.Coder<T> coder) Registers the outbound data endpoint with given Endpoints.ApiServiceDescriptor and Coder, returns the FnDataReceiver responsible for sending the outbound data.
<T> org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.construction.Timer<T>>	addOutgoingTimersEndpoint(java.lang.String timerFamilyId, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.construction.Timer<T>> coder) Registers the outbound timers endpoint with given timer family id and Coder, returns the FnDataReceiver responsible for sending the outbound timers.
<T> void	addPCollectionConsumer(java.lang.String pCollectionId, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer) Register as a consumer for a given PCollection id.
void	addResetFunction(org.apache.beam.sdk.function.ThrowingRunnable resetFunction) Register any reset methods.
void	addStartBundleFunction(org.apache.beam.sdk.function.ThrowingRunnable startBundleFunction) Register any DoFn.StartBundle methods.
void	addTearDownFunction(org.apache.beam.sdk.function.ThrowingRunnable tearDownFunction) Register a tear down handler.
BeamFnDataClient	getBeamFnDataClient() A client for handling inbound and outbound data streams.
BeamFnStateClient	getBeamFnStateClient() A client for handling state requests.
java.util.function.Supplier<Cache<?,?>>	getBundleCacheSupplier() A cache that is used for each bundle and cleared when the bundle completes.
org.apache.beam.sdk.transforms.DoFn.BundleFinalizer	getBundleFinalizer() Register callbacks that will be invoked when the runner completes the bundle.
java.util.function.Supplier<java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleRequest.CacheToken>>	getCacheTokensSupplier() A supplier containing the active cache tokens for this bundle.
java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.Coder>	getCoders() An immutable mapping from coder id to coder definition.
<T> org.apache.beam.sdk.fn.data.FnDataReceiver<T>	getPCollectionConsumer(java.lang.String pCollectionId) Returns a FnDataReceiver to send output to for the specified PCollection id.
java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.PCollection>	getPCollections() An immutable mapping from PCollection id to PCollection definition.
org.apache.beam.sdk.options.PipelineOptions	getPipelineOptions() Pipeline options.
java.util.function.Supplier<java.lang.String>	getProcessBundleInstructionIdSupplier() A supplier containing the active process bundle instruction id.
Cache<?,?>	getProcessWideCache() A cache that is process wide and persists across bundle boundaries.
org.apache.beam.model.pipeline.v1.RunnerApi.PTransform	getPTransform() The PTransform definition.
java.lang.String	getPTransformId() The id of the PTransform.
java.util.Set<java.lang.String>	getRunnerCapabilities() An immutable set of runner capability urns.
org.apache.beam.runners.core.metrics.ShortIdMap	getShortIdMap() A way to get or create monitoring short ids.
BundleSplitListener	getSplitListener() A listener to be invoked when the PTransform splits itself.
java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.WindowingStrategy>	getWindowingStrategies() An immutable mapping from windowing strategy id to windowing strategy definition.

Method Detail

getPipelineOptions

org.apache.beam.sdk.options.PipelineOptions getPipelineOptions()

Pipeline options.

getShortIdMap

org.apache.beam.runners.core.metrics.ShortIdMap getShortIdMap()

A way to get or create monitoring short ids.

getBeamFnDataClient

BeamFnDataClient getBeamFnDataClient()

A client for handling inbound and outbound data streams.

getBeamFnStateClient

BeamFnStateClient getBeamFnStateClient()

A client for handling state requests.

getPTransformId

java.lang.String getPTransformId()

The id of the PTransform.

getPTransform

org.apache.beam.model.pipeline.v1.RunnerApi.PTransform getPTransform()

The PTransform definition.

getProcessBundleInstructionIdSupplier

java.util.function.Supplier<java.lang.String> getProcessBundleInstructionIdSupplier()

A supplier containing the active process bundle instruction id.

getCacheTokensSupplier

java.util.function.Supplier<java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleRequest.CacheToken>> getCacheTokensSupplier()

A supplier containing the active cache tokens for this bundle.

getBundleCacheSupplier

java.util.function.Supplier<Cache<?,?>> getBundleCacheSupplier()

A cache that is used for each bundle and cleared when the bundle completes.

getProcessWideCache

Cache<?,?> getProcessWideCache()

A cache that is process wide and persists across bundle boundaries.

getPCollections

java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.PCollection> getPCollections()

An immutable mapping from PCollection id to PCollection definition.

getCoders

java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.Coder> getCoders()

An immutable mapping from coder id to coder definition.

getWindowingStrategies

java.util.Map<java.lang.String,org.apache.beam.model.pipeline.v1.RunnerApi.WindowingStrategy> getWindowingStrategies()

An immutable mapping from windowing strategy id to windowing strategy definition.

getRunnerCapabilities

java.util.Set<java.lang.String> getRunnerCapabilities()

An immutable set of runner capability urns.

addPCollectionConsumer

<T> void addPCollectionConsumer(java.lang.String pCollectionId,
                                org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer)

Register as a consumer for a given PCollection id.

getPCollectionConsumer

<T> org.apache.beam.sdk.fn.data.FnDataReceiver<T> getPCollectionConsumer(java.lang.String pCollectionId)

Returns a FnDataReceiver to send output to for the specified PCollection id.

addOutgoingDataEndpoint

<T> org.apache.beam.sdk.fn.data.FnDataReceiver<T> addOutgoingDataEndpoint(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                                                          org.apache.beam.sdk.coders.Coder<T> coder)

Registers the outbound data endpoint with given Endpoints.ApiServiceDescriptor and Coder, returns the FnDataReceiver responsible for sending the outbound data.

addOutgoingTimersEndpoint

<T> org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.construction.Timer<T>> addOutgoingTimersEndpoint(java.lang.String timerFamilyId,
                                                                                                                         org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.construction.Timer<T>> coder)

Registers the outbound timers endpoint with given timer family id and Coder, returns the FnDataReceiver responsible for sending the outbound timers.

addStartBundleFunction

void addStartBundleFunction(org.apache.beam.sdk.function.ThrowingRunnable startBundleFunction)

Register any DoFn.StartBundle methods.

addFinishBundleFunction

void addFinishBundleFunction(org.apache.beam.sdk.function.ThrowingRunnable finishBundleFunction)

Register any DoFn.FinishBundle methods.

addIncomingDataEndpoint

<T> void addIncomingDataEndpoint(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                 org.apache.beam.sdk.coders.Coder<T> coder,
                                 org.apache.beam.sdk.fn.data.FnDataReceiver<T> receiver)

addIncomingTimerEndpoint

<T> void addIncomingTimerEndpoint(java.lang.String timerFamilyId,
                                  org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.construction.Timer<T>> coder,
                                  org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.construction.Timer<T>> receiver)

addResetFunction

void addResetFunction(org.apache.beam.sdk.function.ThrowingRunnable resetFunction)

Register any reset methods. This should not invoke any user code which should be done instead using the addFinishBundleFunction(org.apache.beam.sdk.function.ThrowingRunnable). The reset method is guaranteed to be invoked after the bundle completes successfully and after T becomes ineligible to receive requests for monitoring data related to addBundleProgressReporter(org.apache.beam.fn.harness.control.BundleProgressReporter) or getSplitListener().

addTearDownFunction

void addTearDownFunction(org.apache.beam.sdk.function.ThrowingRunnable tearDownFunction)

Register a tear down handler. This method will be invoked before T is eligible to become garbage collected.

addBundleProgressReporter

void addBundleProgressReporter(BundleProgressReporter bundleProgressReporter)

Register a callback whenever progress is being requested.

BundleProgressReporter.updateIntermediateMonitoringData(java.util.Map<java.lang.String, org.apache.beam.vendor.grpc.v1p60p1.com.google.protobuf.ByteString>) will be called by a single arbitrary thread at a time and will be invoked concurrently to the main bundle processing thread. BundleProgressReporter.updateFinalMonitoringData(java.util.Map<java.lang.String, org.apache.beam.vendor.grpc.v1p60p1.com.google.protobuf.ByteString>) will be invoked exclusively by the main bundle processing thread and BundleProgressReporter.updateIntermediateMonitoringData(java.util.Map<java.lang.String, org.apache.beam.vendor.grpc.v1p60p1.com.google.protobuf.ByteString>) will not be invoked until a new bundle starts processing. See BundleProgressReporter for additional details.

getSplitListener

BundleSplitListener getSplitListener()

A listener to be invoked when the PTransform splits itself. This method will be called concurrently to any methods registered with pCollectionConsumerRegistry, startFunctionRegistry, and finishFunctionRegistry.

getBundleFinalizer

org.apache.beam.sdk.transforms.DoFn.BundleFinalizer getBundleFinalizer()

Register callbacks that will be invoked when the runner completes the bundle. The specified instant provides the timeout on how long the finalization callback is valid for.