Package org.apache.beam.runners.spark.translation

Class SparkStreamingTranslationContext

java.lang.Object

org.apache.beam.runners.spark.translation.SparkTranslationContext

org.apache.beam.runners.spark.translation.SparkStreamingTranslationContext

public class SparkStreamingTranslationContext
extends SparkTranslationContext

Translation context used to lazily store Spark datasets during streaming portable pipeline translation and compute them after translation.

Constructor Summary

Constructors

Constructor	Description
SparkStreamingTranslationContext(org.apache.spark.api.java.JavaSparkContext jsc, SparkPipelineOptions options, org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo)

Method Summary

Modifier and Type	Method	Description
org.joda.time.Instant	getFirstTimestamp()
org.apache.spark.streaming.api.java.JavaStreamingContext	getStreamingContext()

Methods inherited from class org.apache.beam.runners.spark.translation.SparkTranslationContext

computeOutputs, getSerializableOptions, getSparkContext, nextSinkId, popDataset, pushDataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkStreamingTranslationContext

public SparkStreamingTranslationContext(org.apache.spark.api.java.JavaSparkContext jsc,
                                        SparkPipelineOptions options,
                                        org.apache.beam.runners.fnexecution.provisioning.JobInfo jobInfo)

Method Detail

getStreamingContext

public org.apache.spark.streaming.api.java.JavaStreamingContext getStreamingContext()

getFirstTimestamp

public org.joda.time.Instant getFirstTimestamp()