All Classes

Class	Description
BoundedDataset<T>	Holds an RDD or values for deferred conversion to an RDD if needed.
BoundedDatasetFactory
ByteArray	Serializable byte array.
CachedSideInputReader	SideInputReader that caches results for costly Materializations.
CachedSideInputReader	SideInputReader that caches materialized views.
Checkpoint	Checkpoint data to make it available in future pipeline runs.
Checkpoint.CheckpointDir	Checkpoint dir tree.
CodahaleCsvSink	A Sink for Spark's metric system reporting metrics (including Beam step metrics) to a CSV file.
CodahaleGraphiteSink	A Sink for Spark's metric system reporting metrics (including Beam step metrics) to Graphite.
CoderHelpers	Serialization utility class.
CoderHelpers	Serialization utility class.
CoderHelpers.FromByteFunction<K,V>	A function for converting a byte array pair to a key-value pair.
ConsoleIO	Print to console.
ConsoleIO.Write	Write to console.
ConsoleIO.Write.Unbound<T>	PTransform writing PCollection to the console.
CreateStream<T>	Create an input stream from Queue.
CsvSink	A Sink for Spark's metric system reporting metrics (including Beam step metrics) to a CSV file.
Dataset	Holder for Spark RDD/DStream.
EmptyCheckpointMark	Passing null values to Spark's Java API may cause problems because of Guava preconditions.
EncoderFactory
EncoderHelpers	Encoders utility class.
EncoderHelpers.Utils	Encoder / expression utils that are called from generated code.
EncoderProvider
EncoderProvider.Factory<T>
EvaluationContext	The EvaluationContext is the result of a pipeline translation and can be used to evaluate / run the pipeline.
EvaluationContext	The EvaluationContext allows us to define pipeline instructions and translate between PObject<T>s or PCollection<T>s and Ts or DStreams/RDDs of Ts.
GlobalWatermarkHolder	A store to hold the global watermarks for a micro-batch.
GlobalWatermarkHolder.SparkWatermarks	A GlobalWatermarkHolder.SparkWatermarks holds the watermarks and batch time relevant to a micro-batch input from a specific source.
GlobalWatermarkHolder.WatermarkAdvancingStreamingListener	Advance the WMs onBatchCompleted event.
GraphiteSink	A Sink for Spark's metric system reporting metrics (including Beam step metrics) to Graphite.
GroupAlsoByWindowViaOutputBufferFn<K,InputT,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow>	A FlatMap function that groups by windows in batch mode using ReduceFnRunner.
GroupCombineFunctions	A set of group/combine functions to apply to Spark RDDs.
GroupNonMergingWindowsFunctions	Functions for GroupByKey with Non-Merging windows translations to Spark.
MetricsAccumulator	For resilience, Accumulators are required to be wrapped in a Singleton.
MetricsAccumulator	AccumulatorV2 for Beam metrics captured in MetricsContainerStepMap.
MetricsAccumulator.AccumulatorCheckpointingSparkListener	Spark Listener which checkpoints MetricsContainerStepMap values for fault-tolerance.
MetricsContainerStepMapAccumulator	AccumulatorV2 implementation for MetricsContainerStepMap.
MicrobatchSource<T,CheckpointMarkT extends org.apache.beam.sdk.io.UnboundedSource.CheckpointMark>	A Source that accommodates Spark's micro-batch oriented nature and wraps an UnboundedSource.
MultiDoFnFunction<InputT,OutputT>	DoFunctions ignore outputs that are not the main output.
NoOpStepContext	A StepContext for Spark Batch Runner execution.
PipelineTranslator	The pipeline translator translates a Beam Pipeline into a Spark correspondence, that can then be evaluated.
PipelineTranslator.TranslationState	Shared, mutable state during the translation of a pipeline and omitted afterwards.
PipelineTranslator.UnresolvedTranslation<InT,T>	Unresolved translation, allowing to optimize the generated Spark DAG.
PipelineTranslatorBatch	PipelineTranslator for executing a Pipeline in Spark in batch mode.
ReifyTimestampsAndWindowsFunction<K,V>	Simple Function to bring the windowing information into the value from the implicit background representation of the PCollection.
ScalaInterop	Utilities for easier interoperability with the Spark Scala API.
ScalaInterop.Fun1<T,V>
ScalaInterop.Fun2<T1,T2,V>
SideInputBroadcast<T>	Broadcast helper for side inputs.
SideInputValues<T>	SideInputValues serves as a Kryo serializable container that contains a materialized view of side inputs.
SideInputValues.BaseSideInputValues<BinaryT,ValuesT extends @NonNull java.lang.Object,T>
SideInputValues.ByWindow<T>	General SideInputValues for BoundedWindows in two possible states.
SideInputValues.Global<T>	Specialized SideInputValues for use with the GlobalWindow in two possible states.
SideInputValues.Loader<T>	Factory function for load SideInputValues from a Dataset.
SourceRDD	Classes implementing Beam Source RDDs.
SourceRDD.Bounded<T>	A SourceRDD.Bounded reads input from a BoundedSource and creates a Spark RDD.
SourceRDD.Unbounded<T,CheckpointMarkT extends org.apache.beam.sdk.io.UnboundedSource.CheckpointMark>	A SourceRDD.Unbounded is the implementation of a micro-batch in a SourceDStream.
SparkAssignWindowFn<T,W extends org.apache.beam.sdk.transforms.windowing.BoundedWindow>	An implementation of Window.Assign for the Spark runner.
SparkBatchPortablePipelineTranslator	Translates a bounded portable pipeline into a Spark job.
SparkBatchPortablePipelineTranslator.IsSparkNativeTransform	Predicate to determine whether a URN is a Spark native transform.
SparkBeamMetricSource	A Spark Source that is tailored to expose a SparkBeamMetric, wrapping an underlying MetricResults instance.
SparkBeamMetricSource	A Spark Source that is tailored to expose a SparkBeamMetric, wrapping an underlying MetricResults instance.
SparkCombineFn<InputT,ValueT,AccumT,OutputT>	A CombineFnBase.GlobalCombineFn with a CombineWithContext.Context for the SparkRunner.
SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT extends SparkCombineFn.WindowedAccumulator<InputT,ValueT,AccumT,ImplT>>	Accumulator of WindowedValues holding values for different windows.
SparkCombineFn.WindowedAccumulator.Type	Type of the accumulator.
SparkCommonPipelineOptions	Spark runner PipelineOptions handles Spark execution-related configurations, such as the master address, and other user-related knobs.
SparkCommonPipelineOptions.StorageLevelFactory	Returns Spark's default storage level for the Dataset or RDD API based on the respective runner.
SparkCommonPipelineOptions.TmpCheckpointDirFactory	Returns the default checkpoint directory of /tmp/${job.name}.
SparkContextFactory
SparkContextOptions	A custom PipelineOptions to work with properties related to JavaSparkContext.
SparkContextOptions.EmptyListenersList	Returns an empty list, to avoid handling null.
SparkExecutableStageContextFactory	Singleton class that contains one ExecutableStageContext.Factory per job.
SparkGroupAlsoByWindowViaWindowSet	An implementation of GroupByKeyViaGroupByKeyOnly.GroupAlsoByWindow logic for grouping by windows and controlling trigger firings and pane accumulation.
SparkGroupAlsoByWindowViaWindowSet.StateAndTimers	State and Timers wrapper.
SparkJobInvoker	Creates a job invocation to manage the Spark runner's execution of a portable pipeline.
SparkJobServerDriver	Driver program that starts a job server for the Spark runner.
SparkJobServerDriver.SparkServerConfiguration	Spark runner-specific Configuration for the jobServer.
SparkNativePipelineVisitor	Pipeline visitor for translating a Beam pipeline into equivalent Spark operations.
SparkPCollectionView	SparkPCollectionView is used to pass serialized views to lambdas.
SparkPipelineOptions	Spark runner PipelineOptions handles Spark execution-related configurations, such as the master address, batch-interval, and other user-related knobs.
SparkPipelineResult	Represents a Spark pipeline execution result.
SparkPipelineRunner	Runs a portable pipeline on Apache Spark.
SparkPipelineTranslator	Translator to support translation between Beam transformations and Spark transformations.
SparkPortablePipelineTranslator<T extends SparkTranslationContext>	Interface for portable Spark translators.
SparkPortableStreamingPipelineOptions	Pipeline options specific to the Spark portable runner running a streaming job.
SparkRunner	The SparkRunner translate operations defined on a pipeline to a representation executable by Spark, and then submitting the job to Spark to be executed.
SparkRunner.Evaluator	Evaluator on the pipeline.
SparkRunnerDebugger	Pipeline runner which translates a Beam pipeline into equivalent Spark operations, without running them.
SparkRunnerDebugger.DebugSparkPipelineResult	PipelineResult of running a Pipeline using SparkRunnerDebugger Use SparkRunnerDebugger.DebugSparkPipelineResult.getDebugString() to get a String representation of the Pipeline translated into Spark native operations.
SparkRunnerKryoRegistrator	Custom KryoRegistrators for Beam's Spark runner needs and registering used class in spark translation for better serialization performance.
SparkRunnerRegistrar	Contains the PipelineRunnerRegistrar and PipelineOptionsRegistrar for the SparkRunner.
SparkRunnerRegistrar.Options	Registers the SparkPipelineOptions.
SparkRunnerRegistrar.Runner	Registers the SparkRunner.
SparkRunnerStreamingContextFactory	A JavaStreamingContext factory for resilience.
SparkSessionFactory
SparkSessionFactory.SparkKryoRegistrator	KryoRegistrator for Spark to serialize broadcast variables used for side-inputs.
SparkSideInputReader	SideInputReader using broadcasted SideInputValues.
SparkSideInputReader	A SideInputReader for the SparkRunner.
SparkStreamingPortablePipelineTranslator	Translates an unbounded portable pipeline into a Spark job.
SparkStreamingTranslationContext	Translation context used to lazily store Spark datasets during streaming portable pipeline translation and compute them after translation.
SparkStructuredStreamingPipelineOptions	Spark runner PipelineOptions handles Spark execution-related configurations, such as the master address, and other user-related knobs.
SparkStructuredStreamingPipelineResult
SparkStructuredStreamingRunner	A Spark runner build on top of Spark's SQL Engine (Structured Streaming framework).
SparkStructuredStreamingRunnerRegistrar	Contains the PipelineRunnerRegistrar and PipelineOptionsRegistrar for the SparkStructuredStreamingRunner.
SparkStructuredStreamingRunnerRegistrar.Options	Registers the SparkStructuredStreamingPipelineOptions.
SparkStructuredStreamingRunnerRegistrar.Runner	Registers the SparkStructuredStreamingRunner.
SparkTimerInternals	An implementation of TimerInternals for the SparkRunner.
SparkTransformOverrides	PTransform overrides for Spark runner.
SparkTranslationContext	Translation context used to lazily store Spark data sets during portable pipeline translation and compute them after translation.
SparkUnboundedSource	A "composite" InputDStream implementation for UnboundedSources.
SparkUnboundedSource.Metadata	A metadata holder for an input stream partition.
StateSpecFunctions	A class containing StateSpec mappingFunctions.
StreamingTransformTranslator	Supports translation between a Beam transform, and Spark's operations on DStreams.
StreamingTransformTranslator.SparkTransformsRegistrar	Registers classes specialized by the Spark runner.
StreamingTransformTranslator.Translator	Translator matches Beam transformation with the appropriate evaluator.
TestDStream<T>
TestSparkPipelineOptions	A SparkPipelineOptions for tests.
TestSparkPipelineOptions.DefaultStopPipelineWatermarkFactory	A factory to provide the default watermark to stop a pipeline that reads from an unbounded source.
TestSparkRunner	The SparkRunner translate operations defined on a pipeline to a representation executable by Spark, and then submitting the job to Spark to be executed.
TimerUtils
TransformEvaluator<TransformT extends org.apache.beam.sdk.transforms.PTransform<?,?>>	Describe a PTransform evaluator.
TransformTranslator<InT extends org.apache.beam.sdk.values.PInput,OutT extends org.apache.beam.sdk.values.POutput,TransformT extends org.apache.beam.sdk.transforms.PTransform<InT,OutT>>	A TransformTranslator provides the capability to translate a specific primitive or composite PTransform into its Spark correspondence.
TransformTranslator	Supports translation between a Beam transform, and Spark's operations on RDDs.
TransformTranslator.Translator	Translator matches Beam transformation with the appropriate evaluator.
TranslationUtils	A set of utilities to help translating Beam transformations into Spark transformations.
TranslationUtils.CombineGroupedValues<K,InputT,OutputT>	A SparkCombineFn function applied to grouped KVs.
TranslationUtils.TupleTagFilter<V>	A utility class to filter TupleTags.
UnboundedDataset<T>	DStream holder Can also crate a DStream from a supplied queue of values, but mainly for testing.
ValueAndCoderKryoSerializer<T>	Kryo serializer for ValueAndCoderLazySerializable.
ValueAndCoderLazySerializable<T>	A holder object that lets you serialize an element with a Coder with minimal wasted space.
WithMetricsSupport	A MetricRegistry decorator-like that supports AggregatorMetric and SparkBeamMetric as Gauges.
WithMetricsSupport	A MetricRegistry decorator-like that supports BeamMetricSets as Gauges.
WordCount	Duplicated from beam-examples-java to avoid dependency.
WordCount	Duplicated from beam-examples-java to avoid dependency.
WordCount.CountWords	A PTransform that converts a PCollection containing lines of text into a PCollection of formatted word counts.
WordCount.CountWords	A PTransform that converts a PCollection containing lines of text into a PCollection of formatted word counts.
WordCount.FormatAsTextFn	A SimpleFunction that converts a Word and Count into a printable string.
WordCount.FormatAsTextFn	A SimpleFunction that converts a Word and Count into a printable string.
WordCount.WordCountOptions	Options supported by WordCount.
WordCount.WordCountOptions	Options supported by WordCount.