org.apache.beam.fn.harness.data

Interface BeamFnDataClient

All Known Implementing Classes:

BeamFnDataGrpcClient

public interface BeamFnDataClient

The BeamFnDataClient is able to forward inbound elements to a FnDataReceiver and provide a receiver of outbound elements. Callers can register themselves as receivers for inbound elements or can get a handle for a receiver of outbound elements.

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator	createOutboundAggregator(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, boolean collectElementsIfNoFlushes) Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane.
void	registerReceiver(java.lang.String instructionId, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors, org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver) Registers a receiver for the provided instruction id.
void	unregisterReceiver(java.lang.String instructionId, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors) Receivers are only expected to be unregistered when bundle processing has completed successfully.

Method Detail

registerReceiver

void registerReceiver(java.lang.String instructionId,
                      java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors,
                      org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> receiver)

Registers a receiver for the provided instruction id.

The receiver is not required to be thread safe.

Receivers for successfully processed bundles must be unregistered. See unregisterReceiver(java.lang.String, java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor>) for details.

Any failure during FnDataReceiver.accept(T) will mark the provided instructionId as invalid and will ignore any future data. It is expected that if a bundle fails during processing then the failure will become visible to the BeamFnDataClient during a future FnDataReceiver.accept(T) invocation.

unregisterReceiver

void unregisterReceiver(java.lang.String instructionId,
                        java.util.List<org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor> apiServiceDescriptors)

Receivers are only expected to be unregistered when bundle processing has completed successfully.

It is expected that if a bundle fails during processing then the failure will become visible to the BeamFnDataClient during a future FnDataReceiver.accept(T) invocation.

createOutboundAggregator

org.apache.beam.sdk.fn.data.BeamFnDataOutboundAggregator createOutboundAggregator(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                                                                  java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                                                                  boolean collectElementsIfNoFlushes)

Creates a BeamFnDataOutboundAggregator for buffering and sending outbound data and timers over the data plane. It is important that BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() is called on the returned BeamFnDataOutboundAggregator at the end of each bundle. If collectElementsIfNoFlushes is set to true, BeamFnDataOutboundAggregator.sendOrCollectBufferedDataAndFinishOutboundStreams() returns the buffered elements instead of sending it through the outbound StreamObserver if there's no previous flush.

Closing the returned aggregator signals the end of the streams.

The returned aggregator is not thread safe.