org.apache.beam.runners.samza.runtime

Class PortableBundleManager<OutT>

java.lang.Object

org.apache.beam.runners.samza.runtime.PortableBundleManager<OutT>

Type Parameters:

OutT - output type of the DoFnOp

All Implemented Interfaces:

BundleManager<OutT>

public class PortableBundleManager<OutT>
extends java.lang.Object
implements BundleManager<OutT>

Bundle management for the DoFnOp that handles lifecycle of a bundle. It also serves as a proxy for the DoFnOp to process watermark and decides to 1. Hold watermark if there is at least one bundle in progress. 2. Propagates the watermark to downstream DAG, if all the previous bundles have completed.

This class is not thread safe and the current implementation relies on the assumption that messages are dispatched to BundleManager in a single threaded mode.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.runners.samza.runtime.BundleManager

BundleManager.BundleProgressListener<OutT>

Constructor Summary

Constructors

Constructor and Description
PortableBundleManager(BundleManager.BundleProgressListener<OutT> bundleProgressListener, long maxBundleSize, long maxBundleTimeMs, org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> bundleTimerScheduler, java.lang.String bundleCheckTimerId)

Method Summary

Modifier and Type	Method and Description
void	inconsistentStateCheck()
void	processTimer(KeyedTimerData<java.lang.Void> keyedTimerData, OpEmitter<OutT> emitter) Signals the BundleManager that a timer is up.
void	processWatermark(org.joda.time.Instant watermark, OpEmitter<OutT> emitter) Signals a watermark event arrived.
void	signalFailure(java.lang.Throwable t) Signal the bundle manager to handle failure.
void	tryFinishBundle(OpEmitter<OutT> emitter) Tries to close the bundle, and reset the bundle to an empty state.
void	tryStartBundle() Starts a new bundle if not already started, then adds an element to the existing bundle.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

PortableBundleManager

public PortableBundleManager(BundleManager.BundleProgressListener<OutT> bundleProgressListener,
                             long maxBundleSize,
                             long maxBundleTimeMs,
                             org.apache.samza.operators.Scheduler<KeyedTimerData<java.lang.Void>> bundleTimerScheduler,
                             java.lang.String bundleCheckTimerId)

Method Detail

tryStartBundle

public void tryStartBundle()

Description copied from interface: BundleManager

Starts a new bundle if not already started, then adds an element to the existing bundle.

Specified by:

tryStartBundle in interface BundleManager<OutT>

processWatermark

public void processWatermark(org.joda.time.Instant watermark,
                             OpEmitter<OutT> emitter)

Description copied from interface: BundleManager

Signals a watermark event arrived. The BundleManager will decide if the watermark needs to be processed, and notify the listener if needed.

Specified by:

processWatermark in interface BundleManager<OutT>

processTimer

public void processTimer(KeyedTimerData<java.lang.Void> keyedTimerData,
                         OpEmitter<OutT> emitter)

Description copied from interface: BundleManager

Signals the BundleManager that a timer is up.

Specified by:

processTimer in interface BundleManager<OutT>

signalFailure

public void signalFailure(java.lang.Throwable t)

Signal the bundle manager to handle failure. We discard the output collected as part of processing the current element and reset the bundle count.

Specified by:

signalFailure in interface BundleManager<OutT>

Parameters:

t - failure cause

tryFinishBundle

public void tryFinishBundle(OpEmitter<OutT> emitter)

Description copied from interface: BundleManager

Tries to close the bundle, and reset the bundle to an empty state.

Specified by:

tryFinishBundle in interface BundleManager<OutT>

inconsistentStateCheck

public void inconsistentStateCheck()