io.openlineage.spark.shade.extension.v1

Interface LineageRelationProvider

public interface LineageRelationProvider

Interface for classes implementing `org.apache.spark.sql.sources.RelationProvider`. `RelationProvider` implements `createRelation` with `SQLContext` and `parameters` as arguments. We want this package to not depend on Spark's code which may be different across Spark versions.

We're aiming to have arguments of `getLineageDataset` the same as arguments of `createRelation` within `RelationProvider`. When implementing this method, one can provide two implementations: one with arguments exactly the same as with `RelationProvider`, and other throwing an exception which should never be called.

Method Summary

Modifier and Type	Method and Description
io.openlineage.client.utils.DatasetIdentifier	getLineageDatasetIdentifier(java.lang.String sparkListenerEventName, io.openlineage.client.OpenLineage openLineage, java.lang.Object sqlContext, java.lang.Object parameters)

Method Detail

getLineageDatasetIdentifier

io.openlineage.client.utils.DatasetIdentifier getLineageDatasetIdentifier(java.lang.String sparkListenerEventName,
                                                                          io.openlineage.client.OpenLineage openLineage,
                                                                          java.lang.Object sqlContext,
                                                                          java.lang.Object parameters)