Package org.apache.beam.runners.spark

Class SparkRunner.Evaluator

java.lang.Object

org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

org.apache.beam.runners.spark.SparkRunner.Evaluator

All Implemented Interfaces:

org.apache.beam.sdk.Pipeline.PipelineVisitor

Direct Known Subclasses:

SparkNativePipelineVisitor

Enclosing class:

SparkRunner

public static class SparkRunner.Evaluator
extends org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

Evaluator on the pipeline.

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.beam.sdk.Pipeline.PipelineVisitor

org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior, org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

Field Summary

Fields

Modifier and Type	Field	Description
protected EvaluationContext	ctxt
protected SparkPipelineTranslator	translator

Constructor Summary

Constructors

Constructor	Description
Evaluator(SparkPipelineTranslator translator, EvaluationContext ctxt)

Method Summary

Modifier and Type	Method	Description
org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior	enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
protected org.apache.beam.sdk.values.PCollection.IsBounded	isBoundedCollection(java.util.Collection<org.apache.beam.sdk.values.PCollection<?>> pValues)
protected boolean	shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)
protected <TransformT extends org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PInput,org.apache.beam.sdk.values.POutput>> TransformEvaluator<TransformT>	translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node, TransformT transform)	Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.
void	visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Methods inherited from class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

enterPipeline, getPipeline, leaveCompositeTransform, leavePipeline, visitValue

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

ctxt

protected final EvaluationContext ctxt

translator

protected final SparkPipelineTranslator translator

Constructor Detail

Evaluator

public Evaluator(SparkPipelineTranslator translator,
                 EvaluationContext ctxt)

Method Detail

enterCompositeTransform

public org.apache.beam.sdk.Pipeline.PipelineVisitor.CompositeBehavior enterCompositeTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Specified by:

enterCompositeTransform in interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Overrides:

enterCompositeTransform in class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

shouldDefer

protected boolean shouldDefer(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

visitPrimitiveTransform

public void visitPrimitiveTransform(org.apache.beam.sdk.runners.TransformHierarchy.Node node)

Specified by:

visitPrimitiveTransform in interface org.apache.beam.sdk.Pipeline.PipelineVisitor

Overrides:

visitPrimitiveTransform in class org.apache.beam.sdk.Pipeline.PipelineVisitor.Defaults

translate

protected <TransformT extends org.apache.beam.sdk.transforms.PTransform<? super org.apache.beam.sdk.values.PInput,org.apache.beam.sdk.values.POutput>> TransformEvaluator<TransformT> translate(org.apache.beam.sdk.runners.TransformHierarchy.Node node,
                                                                                                                                                                                                      TransformT transform)

Determine if this Node belongs to a Bounded branch of the pipeline, or Unbounded, and translate with the proper translator.

isBoundedCollection

protected org.apache.beam.sdk.values.PCollection.IsBounded isBoundedCollection(java.util.Collection<org.apache.beam.sdk.values.PCollection<?>> pValues)