OutT - output type of the DoFnOppublic interface BundleManager<OutT>
DoFnOp that handles lifecycle of a bundle. It also serves as a
proxy for the DoFnOp to process watermark and decides to 1. Hold watermark if there is at
least one bundle in progress. 2. Propagates the watermark to downstream DAG, if all the previous
bundles have completed.
A bundle is considered complete only when the outputs corresponding to each element in the bundle have been resolved and the watermark associated with the bundle(if any) is propagated downstream. The output of an element is considered resolved based on the nature of the ParDoFn 1. In case of synchronous ParDo, outputs of the element is resolved immediately after the processElement returns. 2. In case of asynchronous ParDo, outputs of the element is resolved when all the future emitted by the processElement is resolved.
| Modifier and Type | Interface and Description |
|---|---|
static interface |
BundleManager.BundleProgressListener<OutT>
A listener used to track the lifecycle of a bundle.
|
| Modifier and Type | Method and Description |
|---|---|
void |
processTimer(KeyedTimerData<java.lang.Void> keyedTimerData,
OpEmitter<OutT> emitter)
Signals the BundleManager that a timer is up.
|
void |
processWatermark(org.joda.time.Instant watermark,
OpEmitter<OutT> emitter)
Signals a watermark event arrived.
|
void |
signalFailure(java.lang.Throwable t)
Fails the current bundle, throws away the pending output, and resets the bundle to an empty
state.
|
void |
tryFinishBundle(OpEmitter<OutT> emitter)
Tries to close the bundle, and reset the bundle to an empty state.
|
void |
tryStartBundle()
Starts a new bundle if not already started, then adds an element to the existing bundle.
|
void tryStartBundle()
void processWatermark(org.joda.time.Instant watermark,
OpEmitter<OutT> emitter)
watermark - emitter - void processTimer(KeyedTimerData<java.lang.Void> keyedTimerData, OpEmitter<OutT> emitter)
keyedTimerData - emitter - void signalFailure(java.lang.Throwable t)
t - the throwable that caused the failure.