org.apache.beam.fn.harness.data

Class PCollectionConsumerRegistry

java.lang.Object

org.apache.beam.fn.harness.data.PCollectionConsumerRegistry

public class PCollectionConsumerRegistry
extends java.lang.Object

The PCollectionConsumerRegistry is used to maintain a collection of consuming FnDataReceiver for each pCollectionId. Registering with this class allows inserting an element count counter for every pCollection. A combined MultiplexingConsumer (Wrapped with an ElementCountFnDataReceiver) is returned by calling getMultiplexingConsumer.

Constructor Summary

Constructors

Constructor and Description
PCollectionConsumerRegistry(org.apache.beam.runners.core.metrics.MetricsContainerStepMap metricsContainerRegistry, org.apache.beam.runners.core.metrics.ExecutionStateTracker stateTracker)

Method Summary

Modifier and Type	Method and Description
java.util.Map<java.lang.String,org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString>	getExecutionTimeMonitoringData(org.apache.beam.runners.core.metrics.ShortIdMap shortIds)
java.util.List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo>	getExecutionTimeMonitoringInfos()
org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>>	getMultiplexingConsumer(java.lang.String pCollectionId) New consumers should not be register()-ed after calling this method.
java.util.Set<java.lang.String>	keySet()
<T> void	register(java.lang.String pCollectionId, java.lang.String pTransformId, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer, org.apache.beam.sdk.coders.Coder<T> valueCoder) Register the specified consumer to handle the elements in the pCollection associated with pCollectionId.
void	reset() Reset the execution states of the registered functions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PCollectionConsumerRegistry

public PCollectionConsumerRegistry(org.apache.beam.runners.core.metrics.MetricsContainerStepMap metricsContainerRegistry,
                                   org.apache.beam.runners.core.metrics.ExecutionStateTracker stateTracker)

Method Detail

register

public <T> void register(java.lang.String pCollectionId,
                         java.lang.String pTransformId,
                         org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer,
                         org.apache.beam.sdk.coders.Coder<T> valueCoder)

Register the specified consumer to handle the elements in the pCollection associated with pCollectionId. All consumers must be registered before extracting the combined consumer by calling getMultiplexingConsumer(), or an exception will be thrown.

This will cause both Element Count and Process Bundle Execution time metrics to be collected.

Type Parameters:

T - the element type of the PCollection

Parameters:

pCollectionId -

pTransformId -

consumer -

Throws:

java.lang.RuntimeException - if register() is called after getMultiplexingConsumer() is called.

reset

public void reset()

Reset the execution states of the registered functions.

keySet

public java.util.Set<java.lang.String> keySet()

Returns:

the list of pcollection ids.

getMultiplexingConsumer

public org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<?>> getMultiplexingConsumer(java.lang.String pCollectionId)

New consumers should not be register()-ed after calling this method. This will cause a RuntimeException, as this would fail to properly wrap the late-added consumer to track metrics.

Returns:

A FnDataReceiver which directly wraps all the registered consumers.

getExecutionTimeMonitoringInfos

public java.util.List<org.apache.beam.model.pipeline.v1.MetricsApi.MonitoringInfo> getExecutionTimeMonitoringInfos()

Returns:

Execution Time MonitoringInfos based on the tracked start or finish function.

getExecutionTimeMonitoringData

public java.util.Map<java.lang.String,org.apache.beam.vendor.grpc.v1p43p2.com.google.protobuf.ByteString> getExecutionTimeMonitoringData(org.apache.beam.runners.core.metrics.ShortIdMap shortIds)

Returns:

Execution Time Monitoring data based on the tracked start or finish function.