org.apache.beam.fn.harness.state

Class StateFetchingIterators

java.lang.Object

org.apache.beam.fn.harness.state.StateFetchingIterators

public class StateFetchingIterators
extends java.lang.Object

Adapters which convert a logical series of chunks using continuation tokens over the Beam Fn State API into an Iterator of ByteStrings.

Method Summary

Modifier and Type

Method and Description

static <T> org.apache.beam.fn.harness.state.StateFetchingIterators.CachingStateIterable<T>

readAllAndDecodeStartingFrom(Cache<?,?> cache, BeamFnStateClient beamFnStateClient, org.apache.beam.model.fnexecution.v1.BeamFnApi.StateRequest stateRequestForFirstChunk, org.apache.beam.sdk.coders.Coder<T> valueCoder) This adapter handles using the continuation token to provide iteration over all the elements returned by the Beam Fn State API using the supplied state client, state request for the first chunk of the state stream, and a value decoder.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

readAllAndDecodeStartingFrom

public static <T> org.apache.beam.fn.harness.state.StateFetchingIterators.CachingStateIterable<T> readAllAndDecodeStartingFrom(Cache<?,?> cache,
                                                                                                                               BeamFnStateClient beamFnStateClient,
                                                                                                                               org.apache.beam.model.fnexecution.v1.BeamFnApi.StateRequest stateRequestForFirstChunk,
                                                                                                                               org.apache.beam.sdk.coders.Coder<T> valueCoder)

This adapter handles using the continuation token to provide iteration over all the elements returned by the Beam Fn State API using the supplied state client, state request for the first chunk of the state stream, and a value decoder.

The cache's eviction policy will control how much if any the pages are stored in memory and for how long mutations are stored.

Note: Mutation of the iterable only mutates the underlying cache. It is expected that mutations will have been persisted to the runner such that future reads will reflect those changes.

Parameters:

cache - A cache instance used to store pages of elements and any pending requests. The cache should be namespaced for this object to prevent collisions.

beamFnStateClient - A client for handling state requests.

stateRequestForFirstChunk - A fully populated state request for the first (and possibly only) chunk of a state stream. This state request will be populated with a continuation token to request further chunks of the stream if required.

valueCoder - A coder for decoding the state stream.