Class PipelineDataLineage


  • public class PipelineDataLineage
    extends Object
    This class will help calculate and contain the data lineage for all values in the pipeline.
    What we will get is a List of ValueLineage objects for all the values transforms in the pipeline.
    Each of these ValueLineage objects contains a list of all the transforms it passed through.
    As such, it's a hierarchical view of the pipeline.

    This view will allow us to see immediately where a certain value is being manipulated.

    • Constructor Detail

      • PipelineDataLineage

        public PipelineDataLineage​(PipelineMeta pipelineMeta)
    • Method Detail

      • setPipelineMeta

        public void setPipelineMeta​(PipelineMeta pipelineMeta)
      • getValueLineages

        public List<ValueLineage> getValueLineages()
        Returns:
        the valueLineages
      • setValueLineages

        public void setValueLineages​(List<ValueLineage> valueLineages)
        Parameters:
        valueLineages - the valueLineages to set
      • calculateLineage

        public void calculateLineage​(IVariables variables)
                              throws HopTransformException
        Using the pipeline, we will calculate the data lineage for each field in each transform.
        Throws:
        HopTransformException - In case there is an exception calculating the lineage. This is usually caused by unavailable data sources etc.