org.apache.beam.runners.flink.translation.wrappers.streaming.state

Class FlinkStateInternals<K>

java.lang.Object

org.apache.beam.runners.flink.translation.wrappers.streaming.state.FlinkStateInternals<K>

All Implemented Interfaces:

org.apache.beam.runners.core.StateInternals

public class FlinkStateInternals<K>
extends java.lang.Object
implements org.apache.beam.runners.core.StateInternals

StateInternals that uses a Flink KeyedStateBackend to manage state.

Note: In the Flink streaming runner the key is always encoded using an Coder and stored in a ByteBuffer.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	FlinkStateInternals.EarlyBinder Eagerly create user state to work around https://jira.apache.org/jira/browse/FLINK-12653.

Constructor Summary

Constructors

Constructor and Description
FlinkStateInternals(org.apache.flink.runtime.state.KeyedStateBackend<java.nio.ByteBuffer> flinkStateBackend, org.apache.beam.sdk.coders.Coder<K> keyCoder, org.apache.beam.runners.core.construction.SerializablePipelineOptions pipelineOptions)

Method Summary

Modifier and Type	Method and Description
void	addWatermarkHoldUsage(org.joda.time.Instant watermarkHold)
void	clearGlobalState() Allows to clear all state for the global watermark when the maximum watermark arrives.
K	getKey()
java.lang.Long	minWatermarkHoldMs() Returns the minimum over all watermark holds.
void	removeWatermarkHoldUsage(org.joda.time.Instant watermarkHold)
<T extends org.apache.beam.sdk.state.State> T	state(org.apache.beam.runners.core.StateNamespace namespace, org.apache.beam.runners.core.StateTag<T> address, org.apache.beam.sdk.state.StateContext<?> context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.beam.runners.core.StateInternals

state

Constructor Detail

FlinkStateInternals

public FlinkStateInternals(org.apache.flink.runtime.state.KeyedStateBackend<java.nio.ByteBuffer> flinkStateBackend,
                           org.apache.beam.sdk.coders.Coder<K> keyCoder,
                           org.apache.beam.runners.core.construction.SerializablePipelineOptions pipelineOptions)
                    throws java.lang.Exception

Throws:

java.lang.Exception

Method Detail

minWatermarkHoldMs

public java.lang.Long minWatermarkHoldMs()

Returns the minimum over all watermark holds.

getKey

public K getKey()

Specified by:

getKey in interface org.apache.beam.runners.core.StateInternals

state

public <T extends org.apache.beam.sdk.state.State> T state(org.apache.beam.runners.core.StateNamespace namespace,
                                                           org.apache.beam.runners.core.StateTag<T> address,
                                                           org.apache.beam.sdk.state.StateContext<?> context)

Specified by:

state in interface org.apache.beam.runners.core.StateInternals

clearGlobalState

public void clearGlobalState()

Allows to clear all state for the global watermark when the maximum watermark arrives. We do not clean up the global window state via timers which would lead to an unbounded number of keys and cleanup timers. Instead, the cleanup code below should be run when we finally receive the max watermark.

addWatermarkHoldUsage

public void addWatermarkHoldUsage(org.joda.time.Instant watermarkHold)

removeWatermarkHoldUsage

public void removeWatermarkHoldUsage(org.joda.time.Instant watermarkHold)