Package org.apache.beam.runners.spark.structuredstreaming.translation

Internal translators for running Beam pipelines on Spark.

Interface Summary

Interface	Description
PipelineTranslator.TranslationState	Shared, mutable state during the translation of a pipeline and omitted afterwards.
PipelineTranslator.UnresolvedTranslation<InT,T>	Unresolved translation, allowing to optimize the generated Spark DAG.

Class Summary

Class	Description
EvaluationContext	The EvaluationContext is the result of a pipeline translation and can be used to evaluate / run the pipeline.
PipelineTranslator	The pipeline translator translates a Beam Pipeline into a Spark correspondence, that can then be evaluated.
SparkSessionFactory
SparkSessionFactory.SparkKryoRegistrator	KryoRegistrator for Spark to serialize broadcast variables used for side-inputs.
TransformTranslator<InT extends org.apache.beam.sdk.values.PInput,OutT extends org.apache.beam.sdk.values.POutput,TransformT extends org.apache.beam.sdk.transforms.PTransform<InT,OutT>>	A TransformTranslator provides the capability to translate a specific primitive or composite PTransform into its Spark correspondence.