org.apache.beam.fn.harness.data

Class QueueingBeamFnDataClient

java.lang.Object

org.apache.beam.fn.harness.data.QueueingBeamFnDataClient

All Implemented Interfaces:

BeamFnDataClient

public class QueueingBeamFnDataClient
extends java.lang.Object
implements BeamFnDataClient

A BeamFnDataClient that queues elements so that they can be consumed and processed in the thread which calls @{link #drainAndBlock}.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	QueueingBeamFnDataClient.QueueingFnDataReceiver<T> The QueueingFnDataReceiver is a a FnDataReceiver used by the QueueingBeamFnDataClient.

Constructor Summary

Constructors

Constructor and Description
QueueingBeamFnDataClient(BeamFnDataClient mainClient)

Method Summary

Modifier and Type	Method and Description
void	drainAndBlock() Drains the internal queue of this class, by waiting for all WindowedValues to be passed to their consumers.
<T> org.apache.beam.sdk.fn.data.InboundDataClient	receive(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, org.apache.beam.sdk.fn.data.LogicalEndpoint inputLocation, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<T>> coder, org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer) Registers the following inbound receiver for the provided instruction id and target.
<T> org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>>	send(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor, org.apache.beam.sdk.fn.data.LogicalEndpoint outputLocation, org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<T>> coder) Creates a CloseableFnDataReceiver using the provided instruction id and target.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

QueueingBeamFnDataClient

public QueueingBeamFnDataClient(BeamFnDataClient mainClient)

Method Detail

receive

public <T> org.apache.beam.sdk.fn.data.InboundDataClient receive(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                                                 org.apache.beam.sdk.fn.data.LogicalEndpoint inputLocation,
                                                                 org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<T>> coder,
                                                                 org.apache.beam.sdk.fn.data.FnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> consumer)

Description copied from interface: BeamFnDataClient

Registers the following inbound receiver for the provided instruction id and target.

The provided coder is used to decode inbound elements. The decoded elements are passed to the provided receiver. Any failure during decoding or processing of the element will complete the returned future exceptionally. On successful termination of the stream, the returned future is completed successfully.

The receiver is not required to be thread safe.

Specified by:

receive in interface BeamFnDataClient

drainAndBlock

public void drainAndBlock()
                   throws java.lang.Exception

Drains the internal queue of this class, by waiting for all WindowedValues to be passed to their consumers. The thread which wishes to process() the elements should call this method, as this will cause the consumers to invoke element processing. All receive() and send() calls must be made prior to calling drainAndBlock, in order to properly terminate.

All InboundDataClients will be failed if processing throws an exception.

This method is NOT thread safe. This should only be invoked by a single thread, and is intended for use with a newly constructed QueueingBeamFnDataClient in ProcessBundleHandler#processBundle(InstructionRequest).

Throws:

java.lang.Exception

send

public <T> org.apache.beam.sdk.fn.data.CloseableFnDataReceiver<org.apache.beam.sdk.util.WindowedValue<T>> send(org.apache.beam.model.pipeline.v1.Endpoints.ApiServiceDescriptor apiServiceDescriptor,
                                                                                                               org.apache.beam.sdk.fn.data.LogicalEndpoint outputLocation,
                                                                                                               org.apache.beam.sdk.coders.Coder<org.apache.beam.sdk.util.WindowedValue<T>> coder)

Description copied from interface: BeamFnDataClient

Creates a CloseableFnDataReceiver using the provided instruction id and target.

The provided coder is used to encode elements on the outbound stream.

Closing the returned receiver signals the end of the stream.

The returned closeable receiver is not thread safe.

Specified by:

send in interface BeamFnDataClient