Package io.openlineage.spark.agent.vendor.redshift.lifecycle

Class RedshiftRelationVisitor<D extends io.openlineage.client.OpenLineage.Dataset>

java.lang.Object
scala.runtime.AbstractPartialFunction<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D>>
io.openlineage.spark.api.QueryPlanVisitor<org.apache.spark.sql.execution.datasources.LogicalRelation,D>
io.openlineage.spark.agent.vendor.redshift.lifecycle.RedshiftRelationVisitor<D>
All Implemented Interfaces:
io.openlineage.spark.api.JobNameSuffixProvider<org.apache.spark.sql.execution.datasources.LogicalRelation>, scala.Function1<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D>>, scala.PartialFunction<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D>>
public class RedshiftRelationVisitor<D extends io.openlineage.client.OpenLineage.Dataset> extends io.openlineage.spark.api.QueryPlanVisitor<org.apache.spark.sql.execution.datasources.LogicalRelation,D>
LogicalPlan visitor that matches SaveIntoDataSourceCommands that use a RedshiftRelation. This function extracts a OpenLineage.Dataset from the Redshift table referenced by the relation.

  • Field Summary

    Fields inherited from class io.openlineage.spark.api.QueryPlanVisitor

    context

    Fields inherited from interface io.openlineage.spark.api.JobNameSuffixProvider

    SUFFIX_DELIMITER

  • Constructor Summary

    Constructors
    Constructor
    Description
    RedshiftRelationVisitor(@NonNull io.openlineage.spark.api.OpenLineageContext context, io.openlineage.spark.api.DatasetFactory<D> factory)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    List<D>
    apply(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan x)
     
    boolean
    isDefinedAt(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)
     
    protected boolean
    isRedshiftClass(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)
     

    Methods inherited from class io.openlineage.spark.api.QueryPlanVisitor

    catalogTableFor, inputDataset, internalClassName, isDefinedAt, jobNameSuffix, outputDataset, toString

    Methods inherited from class scala.runtime.AbstractPartialFunction

    andThen, apply$mcDD$sp, apply$mcDF$sp, apply$mcDI$sp, apply$mcDJ$sp, apply$mcFD$sp, apply$mcFF$sp, apply$mcFI$sp, apply$mcFJ$sp, apply$mcID$sp, apply$mcIF$sp, apply$mcII$sp, apply$mcIJ$sp, apply$mcJD$sp, apply$mcJF$sp, apply$mcJI$sp, apply$mcJJ$sp, apply$mcVD$sp, apply$mcVF$sp, apply$mcVI$sp, apply$mcVJ$sp, apply$mcZD$sp, apply$mcZF$sp, apply$mcZI$sp, apply$mcZJ$sp, applyOrElse, compose, lift, orElse, runWith

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    Methods inherited from interface io.openlineage.spark.api.JobNameSuffixProvider

    jobNameSuffix, tableIdentifierToSuffix, trimPath

  • Constructor Details

    • RedshiftRelationVisitor

      public RedshiftRelationVisitor(@NonNull @NonNull io.openlineage.spark.api.OpenLineageContext context, io.openlineage.spark.api.DatasetFactory<D> factory)

  • Method Details

    • apply

      public List<D> apply(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan x)
      Specified by:
      apply in interface scala.Function1<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D extends io.openlineage.client.OpenLineage.Dataset>>
      Overrides:
      apply in class scala.runtime.AbstractPartialFunction<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D extends io.openlineage.client.OpenLineage.Dataset>>

    • isRedshiftClass

      protected boolean isRedshiftClass(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)

    • isDefinedAt

      public boolean isDefinedAt(org.apache.spark.sql.catalyst.plans.logical.LogicalPlan plan)
      Specified by:
      isDefinedAt in interface scala.PartialFunction<org.apache.spark.sql.catalyst.plans.logical.LogicalPlan,List<D extends io.openlineage.client.OpenLineage.Dataset>>
      Overrides:
      isDefinedAt in class io.openlineage.spark.api.QueryPlanVisitor<org.apache.spark.sql.execution.datasources.LogicalRelation,D extends io.openlineage.client.OpenLineage.Dataset>