Package io.openlineage.spark.agent.util

Class RemovePathPatternUtils

java.lang.Object

io.openlineage.spark.agent.util.RemovePathPatternUtils

public class RemovePathPatternUtils
extends Object

Utility class to handle removing path patterns in dataset names. Given a configured regex pattern with "remove" group defined, class methods run regex replacements on all the datasets available within the event

Field Summary

Fields

Modifier and Type	Field	Description
static final String	REMOVE_PATTERN_GROUP
static final String	SPARK_OPENLINEAGE_DATASET_REMOVE_PATH_PATTERN

Constructor Summary

Constructors

Constructor	Description
RemovePathPatternUtils()

Method Summary

Modifier and Type	Method	Description
static List<io.openlineage.client.OpenLineage.InputDataset>	removeInputsPathPattern(io.openlineage.spark.api.OpenLineageContext context, List<io.openlineage.client.OpenLineage.InputDataset> inputs)
static List<io.openlineage.client.OpenLineage.OutputDataset>	removeOutputsPathPattern(io.openlineage.spark.api.OpenLineageContext context, List<io.openlineage.client.OpenLineage.OutputDataset> outputs)
static List<io.openlineage.client.OpenLineage.OutputDataset>	removeOutputsPathPattern_ol(io.openlineage.spark.api.OpenLineageContext context, List<io.openlineage.client.OpenLineage.OutputDataset> outputs)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

REMOVE_PATTERN_GROUP

public static final String REMOVE_PATTERN_GROUP

See Also:

• Constant Field Values

SPARK_OPENLINEAGE_DATASET_REMOVE_PATH_PATTERN

public static final String SPARK_OPENLINEAGE_DATASET_REMOVE_PATH_PATTERN

See Also:

• Constant Field Values

Constructor Details

RemovePathPatternUtils

public RemovePathPatternUtils()

Method Details

removeOutputsPathPattern_ol

public static List<io.openlineage.client.OpenLineage.OutputDataset> removeOutputsPathPattern_ol(io.openlineage.spark.api.OpenLineageContext context,
 List<io.openlineage.client.OpenLineage.OutputDataset> outputs)

removeOutputsPathPattern

public static List<io.openlineage.client.OpenLineage.OutputDataset> removeOutputsPathPattern(io.openlineage.spark.api.OpenLineageContext context,
 List<io.openlineage.client.OpenLineage.OutputDataset> outputs)

removeInputsPathPattern

public static List<io.openlineage.client.OpenLineage.InputDataset> removeInputsPathPattern(io.openlineage.spark.api.OpenLineageContext context,
 List<io.openlineage.client.OpenLineage.InputDataset> inputs)