org.apache.beam.fn.harness.data

Class PCollectionConsumerRegistry

java.lang.Object

org.apache.beam.fn.harness.data.PCollectionConsumerRegistry

public class PCollectionConsumerRegistry
extends java.lang.Object

The PCollectionConsumerRegistry is used to maintain a collection of consuming FnDataReceiver for each pCollectionId. Registering with this class allows inserting an element count counter for every pCollection. A combined MultiplexingConsumer (Wrapped with an ElementCountFnDataReceiver) is returned by calling getMultiplexingConsumer.

Constructor Summary

Constructors

Constructor and Description
PCollectionConsumerRegistry(ExecutionStateSampler.ExecutionStateTracker stateTracker, org.apache.beam.runners.core.metrics.ShortIdMap shortIdMap, BundleProgressReporter.Registrar bundleProgressReporterRegistrar, org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor processBundleDescriptor)
PCollectionConsumerRegistry(ExecutionStateSampler.ExecutionStateTracker stateTracker, org.apache.beam.runners.core.metrics.ShortIdMap shortIdMap, BundleProgressReporter.Registrar bundleProgressReporterRegistrar, org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor processBundleDescriptor, DataSampler dataSampler)

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>>	getMultiplexingConsumer(java.lang.String pCollectionId) New consumers should not be register()-ed after calling this method.
<T> void	register(java.lang.String pCollectionId, java.lang.String pTransformId, java.lang.String pTransformUniqueName, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer) Register the specified consumer to handle the elements in the pCollection associated with pCollectionId.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PCollectionConsumerRegistry

public PCollectionConsumerRegistry(ExecutionStateSampler.ExecutionStateTracker stateTracker,
                                   org.apache.beam.runners.core.metrics.ShortIdMap shortIdMap,
                                   BundleProgressReporter.Registrar bundleProgressReporterRegistrar,
                                   org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor processBundleDescriptor)

PCollectionConsumerRegistry

public PCollectionConsumerRegistry(ExecutionStateSampler.ExecutionStateTracker stateTracker,
                                   org.apache.beam.runners.core.metrics.ShortIdMap shortIdMap,
                                   BundleProgressReporter.Registrar bundleProgressReporterRegistrar,
                                   org.apache.beam.model.fnexecution.v1.BeamFnApi.ProcessBundleDescriptor processBundleDescriptor,
                                   @Nullable
                                   DataSampler dataSampler)

Method Detail

register

public <T> void register(java.lang.String pCollectionId,
                         java.lang.String pTransformId,
                         java.lang.String pTransformUniqueName,
                         org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer)

Register the specified consumer to handle the elements in the pCollection associated with pCollectionId. All consumers must be registered before extracting the combined consumer by calling getMultiplexingConsumer(), or an exception will be thrown.

This will cause both Element Count and Process Bundle Execution time metrics to be collected.

Type Parameters:

T - the element type of the PCollection

Parameters:

pCollectionId -

pTransformId -

pTransformUniqueName -

consumer -

Throws:

java.lang.RuntimeException - if register() is called after getMultiplexingConsumer() is called.

getMultiplexingConsumer

public org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>> getMultiplexingConsumer(java.lang.String pCollectionId)

New consumers should not be register()-ed after calling this method. This will cause a RuntimeException, as this would fail to properly wrap the late-added consumer to track metrics.

Returns:

A FnDataReceiver which directly wraps all the registered consumers.