Package org.apache.beam.sdk.fn.data

Class BeamFnDataOutboundAggregator

  • @NotThreadSafe
public class BeamFnDataOutboundAggregator
extends java.lang.Object
    An outbound data buffering aggregator with size-based buffer and time-based buffer if corresponding options are set.

    The default size-based buffer threshold can be overridden by specifying the experiment data_buffer_size_limit=<bytes>

    The default time-based buffer threshold can be overridden by specifying the experiment data_buffer_time_limit_ms=<milliseconds>

    • Constructor Summary

      Constructors 
      Constructor Description
      BeamFnDataOutboundAggregator​(org.apache.beam.sdk.options.PipelineOptions options, java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier, org.apache.beam.vendor.grpc.v1p48p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver, boolean collectElementsIfNoFlushes)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void discard()  
      <T> FnDataReceiver<T> registerOutputDataLocation​(java.lang.String pTransformId, org.apache.beam.sdk.coders.Coder<T> coder)
      Register the outbound data logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound data.
      <T> FnDataReceiver<T> registerOutputTimersLocation​(java.lang.String pTransformId, java.lang.String timerFamilyId, org.apache.beam.sdk.coders.Coder<T> coder)
      Register the outbound timers logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound timers data.
      void sendElements​(org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements elements)  
      org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements sendOrCollectBufferedDataAndFinishOutboundStreams()
      Closes the streams for all registered outbound endpoints.
      void start()
      Starts the flushing daemon thread if data_buffer_time_limit_ms is set.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • DATA_BUFFER_SIZE_LIMIT

        public static final java.lang.String DATA_BUFFER_SIZE_LIMIT
        See Also:
        Constant Field Values

      • DEFAULT_BUFFER_LIMIT_BYTES

        public static final int DEFAULT_BUFFER_LIMIT_BYTES
        See Also:
        Constant Field Values

      • DATA_BUFFER_TIME_LIMIT_MS

        public static final java.lang.String DATA_BUFFER_TIME_LIMIT_MS
        See Also:
        Constant Field Values

      • DEFAULT_BUFFER_LIMIT_TIME_MS

        public static final long DEFAULT_BUFFER_LIMIT_TIME_MS
        See Also:
        Constant Field Values

    • Constructor Detail

      • BeamFnDataOutboundAggregator

        public BeamFnDataOutboundAggregator​(org.apache.beam.sdk.options.PipelineOptions options,
                                    java.util.function.Supplier<java.lang.String> processBundleRequestIdSupplier,
                                    org.apache.beam.vendor.grpc.v1p48p1.io.grpc.stub.StreamObserver<org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements> outboundObserver,
                                    boolean collectElementsIfNoFlushes)

    • Method Detail

      • start

        public void start()
        Starts the flushing daemon thread if data_buffer_time_limit_ms is set.

      • registerOutputDataLocation

        public <T> FnDataReceiver<T> registerOutputDataLocation​(java.lang.String pTransformId,
                                                        org.apache.beam.sdk.coders.Coder<T> coder)
        Register the outbound data logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound data.

      • registerOutputTimersLocation

        public <T> FnDataReceiver<T> registerOutputTimersLocation​(java.lang.String pTransformId,
                                                          java.lang.String timerFamilyId,
                                                          org.apache.beam.sdk.coders.Coder<T> coder)
        Register the outbound timers logical endpoint, returns the FnDataReceiver for processing the endpoint's outbound timers data.

      • sendOrCollectBufferedDataAndFinishOutboundStreams

        public org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements sendOrCollectBufferedDataAndFinishOutboundStreams()
        Closes the streams for all registered outbound endpoints. Should be called at the end of each bundle. Returns the buffered Elements if the BeamFnDataOutboundAggregator started with collectElementsIfNoFlushes=true, and there was no previous flush in this bundle, otherwise returns null.

      • sendElements

        public void sendElements​(org.apache.beam.model.fnexecution.v1.BeamFnApi.Elements elements)

      • discard

        public void discard()