Package org.apache.hop.pipeline
Class PipelineMeta
- java.lang.Object
-
- org.apache.hop.base.AbstractMeta
-
- org.apache.hop.pipeline.PipelineMeta
-
- All Implemented Interfaces:
Cloneable,Comparable<PipelineMeta>,Comparator<PipelineMeta>,IChanged,IHasFilename,IUndo,IAttributes,IEngineMeta,INamedParameterDefinitions,IXml,IResourceExport
public class PipelineMeta extends AbstractMeta implements IXml, Comparator<PipelineMeta>, Comparable<PipelineMeta>, Cloneable, IResourceExport, IHasFilename
This class defines information about a pipeline and offers methods to save and load it from XML as well as methods to alter a pipeline by adding/removing databases, transforms, hops, etc.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classPipelineMeta.PipelineTypeThe PipelineType enum describes the various types of pipelines in terms of execution, including Normal, Serial Single-Threaded, and Single-Threaded.
-
Field Summary
Fields Modifier and Type Field Description static intBORDER_INDENTprotected booleanchangedHopsprotected booleanchangedTransformsIndicators for changes in transforms, databases, hops, and notes.static String[]descTypeUndoA list of localized strings corresponding to string descriptions of the undo/redo actions.protected List<PipelineHopMeta>hopsThe list of hops associated with the pipeline.protected PipelineMetaInfoinfoprotected Map<String,Boolean>loopCacheThe loop cache.static StringPIPELINE_EXTENSIONprotected intpipelineStatusThe status of the pipeline.protected ResultpreviousResultThe previous result.protected Map<String,List<TransformMeta>>previousTransformCacheThe previous transform cacheprotected List<ITransformMetaChangeListener>transformChangeListenersThe list of TransformChangeListenersprotected Map<String,IRowMeta>transformFieldsCacheThe transforms fields cache.protected List<TransformMeta>transformsThe list of transforms associated with the pipeline.static StringXML_TAGA constant specifying the tag value for the XML node of the pipeline.protected static StringXML_TAG_INFOA constant specifying the tag value for the XML node of the pipeline information.static StringXML_TAG_NOTEPADSA constant specifying the tag value for the XML node of the notes.static StringXML_TAG_ORDERA constant specifying the tag value for the XML node of the order of transforms.static StringXML_TAG_PARAMETERSA constant specifying the tag value for the XML node of the pipeline parameters.static StringXML_TAG_TRANSFORM_ERROR_HANDLINGA constant specifying the tag value for the XML node of the transforms' error-handling information.-
Fields inherited from class org.apache.hop.base.AbstractMeta
attributesMap, changedFlag, changedNotes, contentChangedListeners, currentDirectoryChangedListeners, filename, filenameChangedListeners, maxUndo, metadataProvider, nameChangedListeners, namedParams, notes, runOptions, TYPE_UNDO_CHANGE, TYPE_UNDO_DELETE, TYPE_UNDO_NEW, TYPE_UNDO_POSITION, undo, undoPosition
-
-
Constructor Summary
Constructors Constructor Description PipelineMeta()Builds a new empty pipeline.PipelineMeta(InputStream xmlStream, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace)Instantiates a new pipeline meta-data object.PipelineMeta(String fname, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace)Parses a file containing the XML that describes the pipeline.PipelineMeta(Node pipelineNode, IHopMetadataProvider metadataProvider)Parse a file containing the XML that describes the pipeline.
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidaddMissingPipeline(Missing pipeline)voidaddOrReplaceTransform(TransformMeta transformMeta)Add a new transform to the pipeline if that transform didn't exist yet.voidaddPipelineHop(int p, PipelineHopMeta hi)Add a new hop to the pipeline on a certain location (i.e.voidaddPipelineHop(PipelineHopMeta hi)Add a new hop to the pipeline.voidaddTransform(int p, TransformMeta transformMeta)Add a new transform to the pipeline at the specified index.voidaddTransform(TransformMeta transformMeta)Add a new transform to the pipeline.voidaddTransformChangeListener(int p, ITransformMetaChangeListener list)voidaddTransformChangeListener(ITransformMetaChangeListener listener)voidanalyseImpact(IVariables variables, List<DatabaseImpact> impact, IProgressMonitor monitor)Determines the impact of the different transforms in a pipeline on databases, tables and field.voidcancelQueries()Cancels queries opened for checking & fieldprediction.voidcheckRowMixingStatically(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor)Check a transform to see if there are no multiple transforms to read from.voidcheckTransforms(List<ICheckResult> remarks, boolean onlySelected, IProgressMonitor monitor, IVariables variables, IHopMetadataProvider metadataProvider)Checks all the transforms and fills a List of (CheckResult) remarks.voidclear()Clears the pipeline's meta-data, including the lists of databases, transforms, hops, notes, dependencies, partition schemas, hop servers, and cluster schemas.voidclearCaches()Clears the transform fields and loop caches.voidclearChanged()Clears the different changed flags of the pipeline.Objectclone()Clones the pipeline meta-data object.intcompare(PipelineMeta t1, PipelineMeta t2)Compares two pipeline on name and filename.intcompareTo(PipelineMeta o)Compares this pipeline's meta-data to the specified pipeline's meta-data.booleancontainsTransformMeta(TransformMeta transformMeta)booleanequals(Object obj)Checks whether this pipeline's meta-data object is equal to the specified object.StringexportResources(IVariables variables, Map<String,ResourceDefinition> definitions, IResourceNaming iResourceNaming, IHopMetadataProvider metadataProvider)Exports the specified objects to a flat-file system, adding content with filename keys to a set of definitions.List<PipelineHopMeta>findAllPipelineHopFrom(TransformMeta fromTransform)TransformMetafindMappingInputTransform(String transformName)Finds the mapping input transform with the specified name.TransformMetafindMappingOutputTransform(String transformName)Finds the mapping output transform with the specified name.List<TransformMeta>findNextTransforms(TransformMeta transformMeta)Retrieve a list of succeeding transforms for a certain originating transform.intfindNrInfoTransforms(TransformMeta transformMeta)Find the the number of informational transforms for a certain transform.PipelineHopMetafindPipelineHop(String name)Searches the list of hops for a hop with a certain name.PipelineHopMetafindPipelineHop(PipelineHopMeta hi)Find a certain hop in the pipeline.PipelineHopMetafindPipelineHop(TransformMeta from, TransformMeta to)Search all hops for a hop where a certain transform is at the start and another is at the end.PipelineHopMetafindPipelineHop(TransformMeta from, TransformMeta to, boolean disabledToo)Search all hops for a hop where a certain transform is at the start and another is at the end.PipelineHopMetafindPipelineHopFrom(TransformMeta fromTransform)Search all hops for a hop where a certain transform is at the start.PipelineHopMetafindPipelineHopTo(TransformMeta toTransform)Search all hops for a hop where a certain transform is at the end.booleanfindPrevious(TransformMeta startTransform, TransformMeta transformToFind)Looks in the pipeline to find a transform in a previous location starting somewhere.List<TransformMeta>findPreviousTransforms(TransformMeta transformMeta)Get the list of previous transforms for a certain reference transform.List<TransformMeta>findPreviousTransforms(TransformMeta transformMeta, boolean info)Get the previous transforms on a certain location taking into account the transforms being informational or not.TransformMetafindTransform(String name)Searches the list of transforms for a transform with a certain name.TransformMetafindTransform(String name, TransformMeta exclude)Searches the list of transforms for a transform with a certain name while excluding one transform.StringgetAlternativeTransformName(String transformName)Proposes an alternative transformName when the original already exists.DategetCreatedDate()Gets the date the pipeline was created.StringgetCreatedUser()Gets the user by whom the pipeline was created.StringgetDescription()Gets the description of the workflow.StringgetExtendedDescription()Gets the extended description of the workflow.protected StringgetExtension()List<PipelineHopMeta>getHops()Gets hopsPipelineMetaInfogetInfo()Gets infoTransformMeta[]getInfoTransform(TransformMeta transformMeta)Get the informational transforms for a certain transform.PointgetMaximum()Gets the maximum size of the canvas by calculating the maximum location of a transform.PointgetMinimum()Gets the minimum point on the canvas of a pipeline.List<Missing>getMissingPipeline()DategetModifiedDate()Gets the date the pipeline was modified.StringgetModifiedUser()Gets the user who last modified the pipeline.StringgetName()Get the name of the pipeline.String[]getNextTransformNames(TransformMeta transformMeta)Retrieve an array of succeeding transform names for a certain originating transform.PipelineHopMetagetPipelineHop(int i)Retrieves a hop on a certain location (i.e.List<PipelineHopMeta>getPipelineHops()Get a list of defined hops in this pipeline.List<TransformMeta>getPipelineHopTransforms(boolean all)Gets a List of all the transforms that are used in at least one active hop.intgetPipelineStatus()Gets the status of the pipeline.PipelineMeta.PipelineTypegetPipelineType()Gets the pipeline type.StringgetPipelineVersion()Gets the version of the pipeline.IRowMetagetPrevInfoFields(IVariables variables, String transformName)Find the informational fields coming from an informational transform into the transform specified.IRowMetagetPrevInfoFields(IVariables variables, TransformMeta transformMeta)Find the informational fields coming from an informational transform into the transform specified.IRowMetagetPrevTransformFields(IVariables variables, String transformName)Find the fields that are entering a transform with a certain name.IRowMetagetPrevTransformFields(IVariables variables, TransformMeta transformMeta)Find the fields that are entering a certain transform.IRowMetagetPrevTransformFields(IVariables variables, TransformMeta transformMeta, String transformName, IProgressMonitor monitor)IRowMetagetPrevTransformFields(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor)Find the fields that are entering a certain transform.String[]getPrevTransformNames(String transformName)Retrieve an array of succeeding transform names for a certain originating transform name.String[]getPrevTransformNames(TransformMeta transformMeta)Retrieve an array of preceding transforms for a certain destination transform.TransformMeta[]getPrevTransforms(TransformMeta transformMeta)Retrieve an array of preceding transforms for a certain destination transform.List<ResourceReference>getResourceDependencies(IVariables variables)Gets a list of the resource dependencies.Point[]getSelectedNoteLocations()Get an array of all the selected note locations.Point[]getSelectedTransformLocations()Get an array of all the selected transform locations.String[]getSelectedTransformNames()Gets an array of all the selected transform names.List<TransformMeta>getSelectedTransforms()Gets a list of the selected transforms.List<SqlStatement>getSqlStatements(IVariables variables)Builds a list of all the Sql statements that this pipeline needs in order to work properly.List<SqlStatement>getSqlStatements(IVariables variables, IProgressMonitor monitor)Builds a list of all the Sql statements that this pipeline needs in order to work properly.StringgetSqlStatementsString(IVariables variables)Get the Sql statements (needed to run this pipeline) as a single String.List<StringSearchResult>getStringList(boolean searchTransforms, boolean searchDatabases, boolean searchNotes)Get a list of all the strings used in this pipeline.List<StringSearchResult>getStringList(boolean searchTransforms, boolean searchDatabases, boolean searchNotes, boolean includePasswords)Gets a list of all the strings used in this pipeline.IRowMetagetThisTransformFields(IVariables variables, String transformName, IRowMeta row)Return the fields that are emitted by a transform with a certain name.IRowMetagetThisTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta nextTransform, IRowMeta row)Returns the fields that are emitted by a transform.IRowMetagetThisTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta nextTransform, IRowMeta row, IProgressMonitor monitor)Returns the fields that are emitted by a transform.TransformMetagetTransform(int i)Retrieves a transform on a certain location (i.e.TransformMetagetTransform(int x, int y, int iconsize)Find the transform that is located on a certain point on the canvas, taking into account the icon size.IRowMetagetTransformFields(IVariables variables, String transformName)Returns the fields that are emitted by a certain transform name.IRowMetagetTransformFields(IVariables variables, TransformMeta transformMeta)Returns the fields that are emitted by a certain transform.IRowMetagetTransformFields(IVariables variables, TransformMeta[] transformMeta)Gets the fields for each of the specified transforms and merges them into a single setIRowMetagetTransformFields(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor)Returns the fields that are emitted by a certain transform.IRowMetagetTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta targetTransform, IProgressMonitor monitor)Returns the fields that are emitted by a certain transform.int[]getTransformIndexes(List<TransformMeta> transforms)Gets an array of the locations of an array of transforms.String[]getTransformNames()Gets the names of all the transforms.longgetTransformPerformanceCapturingDelay()Gets the transform performance capturing delay.StringgetTransformPerformanceCapturingSizeLimit()Gets the transform performance capturing size limit.List<TransformMeta>getTransforms()Get a list of defined transforms in this pipeline.TransformMeta[]getTransformsArray()Gets all the transforms as an array.List<TransformMeta>getUsedTransforms()Gets the list of used transforms, which are the transforms that are connected by hops.List<String>getUsedVariables()Gets a list of the used variables in this pipeline.StringgetXml(IVariables variables)Gets the XML representation of this pipeline.booleanhasChanged()Checks whether or not the pipeline has changed.booleanhasLoop(TransformMeta transformMeta)See if there are any loops in the pipeline, starting at the indicated transform.booleanhasLoop(TransformMeta transformMeta, TransformMeta lookup)Checks for loop.booleanhasMissingPlugins()booleanhaveHopsChanged()Checks whether or not any of the hops have been changed.booleanhaveTransformsChanged()Checks whether or not the transforms have changed.intindexOfPipelineHop(PipelineHopMeta hi)Finds the location (index) of the specified hop.intindexOfTransform(TransformMeta transformMeta)Finds the location (index) of the specified transform.booleanisAnySelectedTransformUsedInPipelineHops()Checks if any selected transform has been used in a hop or not.booleanisCapturingTransformPerformanceSnapShots()Checks whether the pipeline is capturing transform performance snapshots.booleanisEmpty()booleanisNameSynchronizedWithFilename()booleanisTransformInformative(TransformMeta thisTransform, TransformMeta prevTransform)Determines whether or not a certain transform is informative.booleanisTransformUsedInPipelineHops(TransformMeta transformMeta)Checks if a transform has been used in a hop or not.booleanisUsingPartitionSchema(PartitionSchema partitionSchema)Checks if the pipeline is using the specified partition schema.voidloadXml(String fname, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace)voidloadXml(Node pipelineNode, String filename, IHopMetadataProvider metadataProvider, IVariables variables)Parses an XML DOM (starting at the specified Node) that describes the pipeline.voidnotifyAllListeners(TransformMeta oldMeta, TransformMeta newMeta)intnrPipelineHops()Gets the number of hops in the pipeline.intnrTransformChangeListeners()Gets the number of transformChangeListeners in the pipeline.intnrTransforms()Gets the number of transforms in the pipeline.booleanpartOfPipelineHop(TransformMeta transformMeta)Determines whether or not a certain transform is part of a hop.ObjectrealClone(boolean doClear)Perform a real clone of the pipeline meta-data object, including cloning all lists and copying all values.voidremoveMissingPipeline(Missing pipeline)voidremovePipelineHop(int i)Removes a hop from the pipeline on a certain location (i.e.voidremovePipelineHop(PipelineHopMeta hop)Removes a hop from the pipeline.voidremoveTransform(int i)Removes a transform from the pipeline on a certain location (i.e.voidremoveTransformChangeListener(ITransformMetaChangeListener list)voidselectAll()Mark all transforms in the pipeline as selected.voidsetCapturingTransformPerformanceSnapShots(boolean capturingTransformPerformanceSnapShots)Sets whether the pipeline is capturing transform performance snapshots.voidsetCreatedDate(Date createdDate)Sets the date the pipeline was created.voidsetCreatedUser(String createdUser)Sets the user by whom the pipeline was created.voidsetDescription(String description)Set the description of the workflow.voidsetExtendedDescription(String extendedDescription)Set the description of the workflow.voidsetHops(List<PipelineHopMeta> hops)Sets hopsvoidsetInfo(PipelineMetaInfo info)Sets infoprotected voidsetInternalEntryCurrentDirectory(IVariables variables)protected voidsetInternalFilenameHopVariables(IVariables variables)Sets the internal filename hop variables.voidsetInternalHopVariables(IVariables variables)Sets the internal hop variables.protected voidsetInternalNameHopVariable(IVariables variables)Sets the internal name hop variable.voidsetModifiedDate(Date modifiedDate)Sets the date the pipeline was modified.voidsetModifiedUser(String modifiedUser)Sets the user who last modified the pipeline.voidsetName(String newName)Set the name.voidsetNameSynchronizedWithFilename(boolean nameSynchronizedWithFilename)voidsetPipelineHop(int i, PipelineHopMeta hi)Changes the content of a hop on a certain position.voidsetPipelineStatus(int pipelineStatus)Sets the status of the pipeline.voidsetPipelineType(PipelineMeta.PipelineType pipelineType)Sets the pipeline type.voidsetPipelineVersion(String pipelineVersion)Sets the version of the pipeline.voidsetTransform(int i, TransformMeta transformMeta)Changes the content of a transform on a certain position.voidsetTransformPerformanceCapturingDelay(long transformPerformanceCapturingDelay)Sets the transform performance capturing delay.voidsetTransformPerformanceCapturingSizeLimit(String transformPerformanceCapturingSizeLimit)Sets the transform performance capturing size limit.voidsetTransforms(List<TransformMeta> transforms)Sets transformsvoidsortHops()Sorts all the hops in the pipeline.voidsortHopsNatural()Sorts the hops in a natural way: from beginning to end.voidsortTransforms()Puts the transforms in alphabetical order.Map<TransformMeta,Map<TransformMeta,Boolean>>sortTransformsNatural()Puts the transforms in a more natural order: from start to finish.StringtoString()Gets a textual representation of the pipeline.voidunselectAll()Clear the selection of all transforms.-
Methods inherited from class org.apache.hop.base.AbstractMeta
addContentChangedListener, addCurrentDirectoryChangedListener, addFilenameChangedListener, addNameChangedListener, addNote, addNote, addObserver, addParameterDefinition, addUndo, canSave, clearNameChangedListeners, clearUndo, compare, deleteObserver, extractNameFromFilename, findDatabase, findDatabase, findHopServer, fireContentChangedListeners, fireContentChangedListeners, fireCurrentDirectoryChanged, fireFilenameChangedListeners, fireNameChangedListeners, getAttribute, getAttributes, getAttributesMap, getContentChangedListeners, getDatabaseNames, getDatabases, getFilename, getHopServerNames, getMaxUndo, getMetadataProvider, getNote, getNote, getNoteIndexes, getNotes, getParameterDefault, getParameterDescription, getSelectedNotes, getUndoSize, hashCode, haveNotesChanged, indexOfNote, isAlwaysShowRunOptions, isClearingLog, isSafeModeEnabled, isShowDialog, listParameters, lowerNote, nameFromFilename, nextUndo, notifyObservers, nrDatabases, nrNotes, previousUndo, raiseNote, removeAllParameters, removeContentChangedListener, removeCurrentDirectoryChangedListener, removeFilenameChangedListener, removeNameChangedListener, removeNote, setAlwaysShowRunOptions, setAttribute, setAttributes, setAttributesMap, setChanged, setChanged, setClearingLog, setFilename, setMaxUndo, setMetadataProvider, setNotes, setSafeModeEnabled, setShowDialog, viewNextUndo, viewPreviousUndo, viewThisUndo
-
Methods inherited from class java.lang.Object
finalize, getClass, notify, notifyAll, wait, wait, wait
-
Methods inherited from interface java.util.Comparator
reversed, thenComparing, thenComparing, thenComparing, thenComparingDouble, thenComparingInt, thenComparingLong
-
Methods inherited from interface org.apache.hop.core.file.IHasFilename
getFilename
-
-
-
-
Field Detail
-
PIPELINE_EXTENSION
public static final String PIPELINE_EXTENSION
- See Also:
- Constant Field Values
-
XML_TAG
public static final String XML_TAG
A constant specifying the tag value for the XML node of the pipeline.- See Also:
- Constant Field Values
-
BORDER_INDENT
public static final int BORDER_INDENT
- See Also:
- Constant Field Values
-
info
protected PipelineMetaInfo info
-
transforms
protected List<TransformMeta> transforms
The list of transforms associated with the pipeline.
-
hops
protected List<PipelineHopMeta> hops
The list of hops associated with the pipeline.
-
pipelineStatus
protected int pipelineStatus
The status of the pipeline.
-
changedTransforms
protected boolean changedTransforms
Indicators for changes in transforms, databases, hops, and notes.
-
changedHops
protected boolean changedHops
-
previousResult
protected Result previousResult
The previous result.
-
transformFieldsCache
protected Map<String,IRowMeta> transformFieldsCache
The transforms fields cache.
-
previousTransformCache
protected Map<String,List<TransformMeta>> previousTransformCache
The previous transform cache
-
transformChangeListeners
protected List<ITransformMetaChangeListener> transformChangeListeners
The list of TransformChangeListeners
-
descTypeUndo
public static final String[] descTypeUndo
A list of localized strings corresponding to string descriptions of the undo/redo actions.
-
XML_TAG_INFO
protected static final String XML_TAG_INFO
A constant specifying the tag value for the XML node of the pipeline information.- See Also:
- Constant Field Values
-
XML_TAG_ORDER
public static final String XML_TAG_ORDER
A constant specifying the tag value for the XML node of the order of transforms.- See Also:
- Constant Field Values
-
XML_TAG_NOTEPADS
public static final String XML_TAG_NOTEPADS
A constant specifying the tag value for the XML node of the notes.- See Also:
- Constant Field Values
-
XML_TAG_PARAMETERS
public static final String XML_TAG_PARAMETERS
A constant specifying the tag value for the XML node of the pipeline parameters.- See Also:
- Constant Field Values
-
XML_TAG_TRANSFORM_ERROR_HANDLING
public static final String XML_TAG_TRANSFORM_ERROR_HANDLING
A constant specifying the tag value for the XML node of the transforms' error-handling information.- See Also:
- Constant Field Values
-
-
Constructor Detail
-
PipelineMeta
public PipelineMeta()
Builds a new empty pipeline. The pipeline will have default logging capability and no variables, and all internal meta-data is cleared to defaults.
-
PipelineMeta
public PipelineMeta(String fname, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace) throws HopXmlException, HopMissingPluginsException
Parses a file containing the XML that describes the pipeline.- Parameters:
fname- The filenamemetadataProvider- the metadata store to reference (or null if there is none)parentVariableSpace- the parent variable variables to use during PipelineMeta construction- Throws:
HopXmlException- if any errors occur during parsing of the specified fileHopMissingPluginsException- in case missing plugins were found (details are in the exception in that case)
-
PipelineMeta
public PipelineMeta(InputStream xmlStream, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace) throws HopXmlException, HopMissingPluginsException
Instantiates a new pipeline meta-data object.- Parameters:
xmlStream- the XML input stream from which to read the pipeline definitionparentVariableSpace- the parent variable variables- Throws:
HopXmlException- if any errors occur during parsing of the specified streamHopMissingPluginsException- in case missing plugins were found (details are in the exception in that case)
-
PipelineMeta
public PipelineMeta(Node pipelineNode, IHopMetadataProvider metadataProvider) throws HopXmlException, HopMissingPluginsException
Parse a file containing the XML that describes the pipeline.- Parameters:
pipelineNode- The XML node to load from- Throws:
HopXmlException- if any errors occur during parsing of the specified fileHopMissingPluginsException- in case missing plugins were found (details are in the exception in that case)
-
-
Method Detail
-
compare
public int compare(PipelineMeta t1, PipelineMeta t2)
Compares two pipeline on name and filename. The comparison algorithm is as follows:
- The first pipeline's filename is checked first; if it has none, the pipeline is generated If the second pipeline is not generated, -1 is returned.
- If the pipelines are both generated, the pipelines' names are compared. If the first pipeline has no name and the second one does, a -1 is returned. If the opposite is true, 1 is returned.
- If they both have names they are compared as strings. If the result is non-zero it is returned. Otherwise the repository directories are compared using the same technique of checking empty values and then performing a string comparison, returning any non-zero result.
- Specified by:
comparein interfaceComparator<PipelineMeta>- Parameters:
t1- the first pipeline to comparet2- the second pipeline to compare- Returns:
- 0 if the two pipelines are equal, 1 or -1 depending on the values (see description above)
-
compareTo
public int compareTo(PipelineMeta o)
Compares this pipeline's meta-data to the specified pipeline's meta-data. This method simply calls compare(this, o)- Specified by:
compareToin interfaceComparable<PipelineMeta>- Parameters:
o- the o- Returns:
- the int
- See Also:
compare(PipelineMeta, PipelineMeta),Comparable.compareTo(Object)
-
equals
public boolean equals(Object obj)
Checks whether this pipeline's meta-data object is equal to the specified object. If the specified object is not an instance of PipelineMeta, false is returned. Otherwise the method returns whether a call to compare() indicates equality (i.e. compare(this, (PipelineMeta)obj)==0).- Specified by:
equalsin interfaceComparator<PipelineMeta>- Overrides:
equalsin classObject- Parameters:
obj- the obj- Returns:
- true, if successful
- See Also:
compare(PipelineMeta, PipelineMeta),Object.equals(Object)
-
clone
public Object clone()
Clones the pipeline meta-data object.- Overrides:
clonein classObject- Returns:
- a clone of the pipeline meta-data object
- See Also:
Object.clone()
-
realClone
public Object realClone(boolean doClear)
Perform a real clone of the pipeline meta-data object, including cloning all lists and copying all values. If the doClear parameter is true, the clone will be cleared of ALL values before the copy. If false, only the copied fields will be cleared.- Parameters:
doClear- Whether to clear all of the clone's data before copying from the source object- Returns:
- a real clone of the calling object
-
getExtension
protected String getExtension()
- Specified by:
getExtensionin classAbstractMeta
-
clear
public void clear()
Clears the pipeline's meta-data, including the lists of databases, transforms, hops, notes, dependencies, partition schemas, hop servers, and cluster schemas. Logging information and timeouts are reset to defaults, and recent connection info is cleared.- Overrides:
clearin classAbstractMeta
-
addTransform
public void addTransform(TransformMeta transformMeta)
Add a new transform to the pipeline. Also marks that the pipeline's transforms have changed.- Parameters:
transformMeta- The meta-data for the transform to be added.
-
addOrReplaceTransform
public void addOrReplaceTransform(TransformMeta transformMeta)
Add a new transform to the pipeline if that transform didn't exist yet. Otherwise, replace the transform. This method also marks that the pipeline's transforms have changed.- Parameters:
transformMeta- The meta-data for the transform to be added.
-
addPipelineHop
public void addPipelineHop(PipelineHopMeta hi)
Add a new hop to the pipeline. The hop information (source and target transforms, e.g.) should be configured in the PipelineHopMeta object before calling addPipelineHop(). Also marks that the pipeline's hops have changed.- Parameters:
hi- The hop meta-data to be added.
-
addTransform
public void addTransform(int p, TransformMeta transformMeta)Add a new transform to the pipeline at the specified index. This method sets the transform's parent pipeline to the this pipeline, and marks that the pipelines' transforms have changed.- Parameters:
p- The index into the transform listtransformMeta- The transform to be added.
-
addPipelineHop
public void addPipelineHop(int p, PipelineHopMeta hi)Add a new hop to the pipeline on a certain location (i.e. the specified index). Also marks that the pipeline's hops have changed.- Parameters:
p- the index into the hop listhi- The hop to be added.
-
getTransforms
public List<TransformMeta> getTransforms()
Get a list of defined transforms in this pipeline.- Returns:
- an ArrayList of defined transforms.
-
getTransform
public TransformMeta getTransform(int i)
Retrieves a transform on a certain location (i.e. the specified index).- Parameters:
i- The index into the transforms list.- Returns:
- The desired transform's meta-data.
-
getPipelineHops
public List<PipelineHopMeta> getPipelineHops()
Get a list of defined hops in this pipeline.- Returns:
- a list of defined hops.
-
getPipelineHop
public PipelineHopMeta getPipelineHop(int i)
Retrieves a hop on a certain location (i.e. the specified index).- Parameters:
i- The index into the hops list.- Returns:
- The desired hop's meta-data.
-
removeTransform
public void removeTransform(int i)
Removes a transform from the pipeline on a certain location (i.e. the specified index). Also marks that the pipeline's transforms have changed.- Parameters:
i- The index
-
removePipelineHop
public void removePipelineHop(int i)
Removes a hop from the pipeline on a certain location (i.e. the specified index). Also marks that the pipeline's hops have changed.- Parameters:
i- The index into the hops list
-
removePipelineHop
public void removePipelineHop(PipelineHopMeta hop)
Removes a hop from the pipeline. Also marks that the pipeline's hops have changed.- Parameters:
hop- The hop to remove from the list of hops
-
nrTransforms
public int nrTransforms()
Gets the number of transforms in the pipeline.- Returns:
- The number of transforms in the pipeline.
-
nrPipelineHops
public int nrPipelineHops()
Gets the number of hops in the pipeline.- Returns:
- The number of hops in the pipeline.
-
nrTransformChangeListeners
public int nrTransformChangeListeners()
Gets the number of transformChangeListeners in the pipeline.- Returns:
- The number of transformChangeListeners in the pipeline.
-
setTransform
public void setTransform(int i, TransformMeta transformMeta)Changes the content of a transform on a certain position. This is accomplished by setting the transform's metadata at the specified index to the specified meta-data object. The new transform's parent pipeline is updated to be this pipeline.- Parameters:
i- The index into the transforms listtransformMeta- The transform meta-data to set
-
setPipelineHop
public void setPipelineHop(int i, PipelineHopMeta hi)Changes the content of a hop on a certain position. This is accomplished by setting the hop's metadata at the specified index to the specified meta-data object.- Parameters:
i- The index into the hops listhi- The hop meta-data to set
-
getUsedTransforms
public List<TransformMeta> getUsedTransforms()
Gets the list of used transforms, which are the transforms that are connected by hops.- Returns:
- a list with all the used transforms
-
findTransform
public TransformMeta findTransform(String name)
Searches the list of transforms for a transform with a certain name.- Parameters:
name- The name of the transform to look for- Returns:
- The transform information or null if no nothing was found.
-
findTransform
public TransformMeta findTransform(String name, TransformMeta exclude)
Searches the list of transforms for a transform with a certain name while excluding one transform.- Parameters:
name- The name of the transform to look forexclude- The transform information to exclude.- Returns:
- The transform information or null if nothing was found.
-
findPipelineHop
public PipelineHopMeta findPipelineHop(String name)
Searches the list of hops for a hop with a certain name.- Parameters:
name- The name of the hop to look for- Returns:
- The hop information or null if nothing was found.
-
findPipelineHopFrom
public PipelineHopMeta findPipelineHopFrom(TransformMeta fromTransform)
Search all hops for a hop where a certain transform is at the start.- Parameters:
fromTransform- The transform at the start of the hop.- Returns:
- The hop or null if no hop was found.
-
findAllPipelineHopFrom
public List<PipelineHopMeta> findAllPipelineHopFrom(TransformMeta fromTransform)
-
findPipelineHop
public PipelineHopMeta findPipelineHop(PipelineHopMeta hi)
Find a certain hop in the pipeline.- Parameters:
hi- The hop information to look for.- Returns:
- The hop or null if no hop was found.
-
findPipelineHop
public PipelineHopMeta findPipelineHop(TransformMeta from, TransformMeta to)
Search all hops for a hop where a certain transform is at the start and another is at the end.- Parameters:
from- The transform at the start of the hop.to- The transform at the end of the hop.- Returns:
- The hop or null if no hop was found.
-
findPipelineHop
public PipelineHopMeta findPipelineHop(TransformMeta from, TransformMeta to, boolean disabledToo)
Search all hops for a hop where a certain transform is at the start and another is at the end.- Parameters:
from- The transform at the start of the hop.to- The transform at the end of the hop.disabledToo- the disabled too- Returns:
- The hop or null if no hop was found.
-
findPipelineHopTo
public PipelineHopMeta findPipelineHopTo(TransformMeta toTransform)
Search all hops for a hop where a certain transform is at the end.- Parameters:
toTransform- The transform at the end of the hop.- Returns:
- The hop or null if no hop was found.
-
isTransformInformative
public boolean isTransformInformative(TransformMeta thisTransform, TransformMeta prevTransform)
Determines whether or not a certain transform is informative. This means that the previous transform is sending information to this transform, but only informative. This means that this transform is using the information to process the actual stream of data. We use this in StreamLookup, TableInput and other types of transforms.- Parameters:
thisTransform- The transform that is receiving information.prevTransform- The transform that is sending information- Returns:
- true if prevTransform if informative for thisTransform.
-
findPreviousTransforms
public List<TransformMeta> findPreviousTransforms(TransformMeta transformMeta)
Get the list of previous transforms for a certain reference transform. This includes the info transforms.- Parameters:
transformMeta- The reference transform- Returns:
- The list of the preceding transforms, including the info transforms.
-
findPreviousTransforms
public List<TransformMeta> findPreviousTransforms(TransformMeta transformMeta, boolean info)
Get the previous transforms on a certain location taking into account the transforms being informational or not.- Parameters:
transformMeta- The name of the transforminfo- true if we only want the informational transforms.- Returns:
- The list of the preceding transforms
-
getInfoTransform
public TransformMeta[] getInfoTransform(TransformMeta transformMeta)
Get the informational transforms for a certain transform. An informational transform is a transform that provides information for lookups, etc.- Parameters:
transformMeta- The name of the transform- Returns:
- An array of the informational transforms found
-
findNrInfoTransforms
public int findNrInfoTransforms(TransformMeta transformMeta)
Find the the number of informational transforms for a certain transform.- Parameters:
transformMeta- The transform- Returns:
- The number of informational transforms found.
-
getPrevInfoFields
public IRowMeta getPrevInfoFields(IVariables variables, String transformName) throws HopTransformException
Find the informational fields coming from an informational transform into the transform specified.- Parameters:
transformName- The name of the transform- Returns:
- A row containing fields with origin.
- Throws:
HopTransformException- the hop transform exception
-
getPrevInfoFields
public IRowMeta getPrevInfoFields(IVariables variables, TransformMeta transformMeta) throws HopTransformException
Find the informational fields coming from an informational transform into the transform specified.- Parameters:
transformMeta- The receiving transform- Returns:
- A row containing fields with origin.
- Throws:
HopTransformException- the hop transform exception
-
getPrevTransforms
public TransformMeta[] getPrevTransforms(TransformMeta transformMeta)
Retrieve an array of preceding transforms for a certain destination transform. This includes the info transforms.- Parameters:
transformMeta- The destination transform- Returns:
- An array containing the preceding transforms.
-
getPrevTransformNames
public String[] getPrevTransformNames(String transformName)
Retrieve an array of succeeding transform names for a certain originating transform name.- Parameters:
transformName- The originating transform name- Returns:
- An array of succeeding transform names
-
getPrevTransformNames
public String[] getPrevTransformNames(TransformMeta transformMeta)
Retrieve an array of preceding transforms for a certain destination transform.- Parameters:
transformMeta- The destination transform- Returns:
- an array of preceding transform names.
-
findNextTransforms
public List<TransformMeta> findNextTransforms(TransformMeta transformMeta)
Retrieve a list of succeeding transforms for a certain originating transform.- Parameters:
transformMeta- The originating transform- Returns:
- an array of succeeding transforms.
-
getNextTransformNames
public String[] getNextTransformNames(TransformMeta transformMeta)
Retrieve an array of succeeding transform names for a certain originating transform.- Parameters:
transformMeta- The originating transform- Returns:
- an array of succeeding transform names.
-
getTransform
public TransformMeta getTransform(int x, int y, int iconsize)
Find the transform that is located on a certain point on the canvas, taking into account the icon size.- Parameters:
x- the x-coordinate of the point queriedy- the y-coordinate of the point queriediconsize- the iconsize- Returns:
- The transform information if a transform is located at the point. Otherwise, if no transform was found: null.
-
partOfPipelineHop
public boolean partOfPipelineHop(TransformMeta transformMeta)
Determines whether or not a certain transform is part of a hop.- Parameters:
transformMeta- The transform queried- Returns:
- true if the transform is part of a hop.
-
getTransformFields
public IRowMeta getTransformFields(IVariables variables, String transformName) throws HopTransformException
Returns the fields that are emitted by a certain transform name.- Parameters:
transformName- The transformName of the transform to be queried.- Returns:
- A row containing the fields emitted.
- Throws:
HopTransformException- the hop transform exception
-
getTransformFields
public IRowMeta getTransformFields(IVariables variables, TransformMeta transformMeta) throws HopTransformException
Returns the fields that are emitted by a certain transform.- Parameters:
transformMeta- The transform to be queried.- Returns:
- A row containing the fields emitted.
- Throws:
HopTransformException- the hop transform exception
-
getTransformFields
public IRowMeta getTransformFields(IVariables variables, TransformMeta[] transformMeta) throws HopTransformException
Gets the fields for each of the specified transforms and merges them into a single set- Parameters:
transformMeta- the transform meta- Returns:
- an interface to the transform fields
- Throws:
HopTransformException- the hop transform exception
-
getTransformFields
public IRowMeta getTransformFields(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor) throws HopTransformException
Returns the fields that are emitted by a certain transform.- Parameters:
transformMeta- The transform to be queried.monitor- The progress monitor for progress dialog. (null if not used!)- Returns:
- A row containing the fields emitted.
- Throws:
HopTransformException- the hop transform exception
-
getTransformFields
public IRowMeta getTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta targetTransform, IProgressMonitor monitor) throws HopTransformException
Returns the fields that are emitted by a certain transform.- Parameters:
transformMeta- The transform to be queried.targetTransform- the target transformmonitor- The progress monitor for progress dialog. (null if not used!)- Returns:
- A row containing the fields emitted.
- Throws:
HopTransformException- the hop transform exception
-
getPrevTransformFields
public IRowMeta getPrevTransformFields(IVariables variables, String transformName) throws HopTransformException
Find the fields that are entering a transform with a certain name.- Parameters:
transformName- The name of the transform queried- Returns:
- A row containing the fields (w/ origin) entering the transform
- Throws:
HopTransformException- the hop transform exception
-
getPrevTransformFields
public IRowMeta getPrevTransformFields(IVariables variables, TransformMeta transformMeta) throws HopTransformException
Find the fields that are entering a certain transform.- Parameters:
transformMeta- The transform queried- Returns:
- A row containing the fields (w/ origin) entering the transform
- Throws:
HopTransformException- the hop transform exception
-
getPrevTransformFields
public IRowMeta getPrevTransformFields(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor) throws HopTransformException
Find the fields that are entering a certain transform.- Parameters:
transformMeta- The transform queriedmonitor- The progress monitor for progress dialog. (null if not used!)- Returns:
- A row containing the fields (w/ origin) entering the transform
- Throws:
HopTransformException- the hop transform exception
-
getPrevTransformFields
public IRowMeta getPrevTransformFields(IVariables variables, TransformMeta transformMeta, String transformName, IProgressMonitor monitor) throws HopTransformException
- Throws:
HopTransformException
-
getThisTransformFields
public IRowMeta getThisTransformFields(IVariables variables, String transformName, IRowMeta row) throws HopTransformException
Return the fields that are emitted by a transform with a certain name.- Parameters:
transformName- The name of the transform that's being queried.row- A row containing the input fields or an empty row if no input is required.- Returns:
- A Row containing the output fields.
- Throws:
HopTransformException- the hop transform exception
-
getThisTransformFields
public IRowMeta getThisTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta nextTransform, IRowMeta row) throws HopTransformException
Returns the fields that are emitted by a transform.- Parameters:
transformMeta- : The TransformMeta object that's being queriednextTransform- : if non-null this is the next transform that's call back to ask what's being sentrow- : A row containing the input fields or an empty row if no input is required.- Returns:
- A Row containing the output fields.
- Throws:
HopTransformException- the hop transform exception
-
getThisTransformFields
public IRowMeta getThisTransformFields(IVariables variables, TransformMeta transformMeta, TransformMeta nextTransform, IRowMeta row, IProgressMonitor monitor) throws HopTransformException
Returns the fields that are emitted by a transform.- Parameters:
transformMeta- : The TransformMeta object that's being queriednextTransform- : if non-null this is the next transform that's call back to ask what's being sentrow- : A row containing the input fields or an empty row if no input is required.monitor- the monitor- Returns:
- A Row containing the output fields.
- Throws:
HopTransformException- the hop transform exception
-
isUsingPartitionSchema
public boolean isUsingPartitionSchema(PartitionSchema partitionSchema)
Checks if the pipeline is using the specified partition schema.- Parameters:
partitionSchema- the partition schema- Returns:
- true if the pipeline is using the partition schema, false otherwise
-
indexOfPipelineHop
public int indexOfPipelineHop(PipelineHopMeta hi)
Finds the location (index) of the specified hop.- Parameters:
hi- The hop queried- Returns:
- The location of the hop, or -1 if nothing was found.
-
indexOfTransform
public int indexOfTransform(TransformMeta transformMeta)
Finds the location (index) of the specified transform.- Parameters:
transformMeta- The transform queried- Returns:
- The location of the transform, or -1 if nothing was found.
-
getXml
public String getXml(IVariables variables) throws HopException
Gets the XML representation of this pipeline.- Specified by:
getXmlin interfaceIEngineMeta- Specified by:
getXmlin interfaceIXml- Parameters:
variables-- Returns:
- the XML representation of this pipeline
- Throws:
HopException- if any errors occur during generation of the XML- See Also:
IXml.getXml(IVariables)
-
loadXml
public void loadXml(String fname, IHopMetadataProvider metadataProvider, IVariables parentVariableSpace) throws HopXmlException, HopMissingPluginsException
-
loadXml
public void loadXml(Node pipelineNode, String filename, IHopMetadataProvider metadataProvider, IVariables variables) throws HopXmlException, HopMissingPluginsException
Parses an XML DOM (starting at the specified Node) that describes the pipeline.- Parameters:
pipelineNode- The XML node to load fromfilename- The filenamevariables- the parent variable variables to use during PipelineMeta construction- Throws:
HopXmlException- if any errors occur during parsing of the specified fileHopMissingPluginsException- in case missing plugins were found (details are in the exception in that case)
-
getPipelineHopTransforms
public List<TransformMeta> getPipelineHopTransforms(boolean all)
Gets a List of all the transforms that are used in at least one active hop. These transforms will be used to execute the pipeline. The others will not be executed.
Update 3.0 : we also add those transforms that are not linked to another hop, but have at least one remote input or output transform defined.- Parameters:
all- true if you want to get ALL the transforms from the pipeline, false otherwise- Returns:
- A List of transforms
-
isTransformUsedInPipelineHops
public boolean isTransformUsedInPipelineHops(TransformMeta transformMeta)
Checks if a transform has been used in a hop or not.- Parameters:
transformMeta- The transform queried.- Returns:
- true if a transform is used in a hop (active or not), false otherwise
-
isAnySelectedTransformUsedInPipelineHops
public boolean isAnySelectedTransformUsedInPipelineHops()
Checks if any selected transform has been used in a hop or not.- Returns:
- true if a transform is used in a hop (active or not), false otherwise
-
clearChanged
public void clearChanged()
Clears the different changed flags of the pipeline.- Specified by:
clearChangedin interfaceIChanged- Specified by:
clearChangedin interfaceIEngineMeta- Overrides:
clearChangedin classAbstractMeta
-
haveTransformsChanged
public boolean haveTransformsChanged()
Checks whether or not the transforms have changed.- Returns:
- true if the transforms have been changed, false otherwise
-
haveHopsChanged
public boolean haveHopsChanged()
Checks whether or not any of the hops have been changed.- Returns:
- true if a hop has been changed, false otherwise
-
hasChanged
public boolean hasChanged()
Checks whether or not the pipeline has changed.- Specified by:
hasChangedin interfaceIChanged- Overrides:
hasChangedin classAbstractMeta- Returns:
- true if the pipeline has changed, false otherwise
-
hasLoop
public boolean hasLoop(TransformMeta transformMeta)
See if there are any loops in the pipeline, starting at the indicated transform. This works by looking at all the previous transforms. If you keep going backward and find the transform, there is a loop. Both the informational and the normal transforms need to be checked for loops!- Parameters:
transformMeta- The transform position to start looking- Returns:
- true if a loop has been found, false if no loop is found.
-
hasLoop
public boolean hasLoop(TransformMeta transformMeta, TransformMeta lookup)
Checks for loop.- Parameters:
transformMeta- the transformmetalookup- the lookup- Returns:
- true, if successful
-
selectAll
public void selectAll()
Mark all transforms in the pipeline as selected.
-
unselectAll
public void unselectAll()
Clear the selection of all transforms.
-
getSelectedTransformLocations
public Point[] getSelectedTransformLocations()
Get an array of all the selected transform locations.- Returns:
- The selected transform locations.
-
getSelectedNoteLocations
public Point[] getSelectedNoteLocations()
Get an array of all the selected note locations.- Returns:
- The selected note locations.
-
getSelectedTransforms
public List<TransformMeta> getSelectedTransforms()
Gets a list of the selected transforms.- Returns:
- A list of all the selected transforms.
-
getSelectedTransformNames
public String[] getSelectedTransformNames()
Gets an array of all the selected transform names.- Returns:
- An array of all the selected transform names.
-
getTransformIndexes
public int[] getTransformIndexes(List<TransformMeta> transforms)
Gets an array of the locations of an array of transforms.- Parameters:
transforms- An array of transforms- Returns:
- an array of the locations of an array of transforms
-
getMaximum
public Point getMaximum()
Gets the maximum size of the canvas by calculating the maximum location of a transform.- Returns:
- Maximum coordinate of a transform in the pipeline + (100,100) for safety.
-
getMinimum
public Point getMinimum()
Gets the minimum point on the canvas of a pipeline.- Returns:
- Minimum coordinate of a transform in the pipeline
-
getTransformNames
public String[] getTransformNames()
Gets the names of all the transforms.- Returns:
- An array of transform names.
-
getTransformsArray
public TransformMeta[] getTransformsArray()
Gets all the transforms as an array.- Returns:
- An array of all the transforms in the pipeline.
-
findPrevious
public boolean findPrevious(TransformMeta startTransform, TransformMeta transformToFind)
Looks in the pipeline to find a transform in a previous location starting somewhere.- Parameters:
startTransform- The starting transformtransformToFind- The transform to look for backward in the pipeline- Returns:
- true if we can find the transform in an earlier location in the pipeline.
-
sortTransforms
public void sortTransforms()
Puts the transforms in alphabetical order.
-
sortHops
public void sortHops()
Sorts all the hops in the pipeline.
-
sortTransformsNatural
public Map<TransformMeta,Map<TransformMeta,Boolean>> sortTransformsNatural()
Puts the transforms in a more natural order: from start to finish. For the moment, we ignore splits and joins. Splits and joins can't be listed sequentially in any case!- Returns:
- a map containing all the previous transforms per transform
-
sortHopsNatural
public void sortHopsNatural()
Sorts the hops in a natural way: from beginning to end.
-
analyseImpact
public void analyseImpact(IVariables variables, List<DatabaseImpact> impact, IProgressMonitor monitor) throws HopTransformException
Determines the impact of the different transforms in a pipeline on databases, tables and field.- Parameters:
impact- An ArrayList of DatabaseImpact objects.monitor- a progress monitor listener to be updated as the pipeline is analyzed- Throws:
HopTransformException- if any errors occur during analysis
-
getAlternativeTransformName
public String getAlternativeTransformName(String transformName)
Proposes an alternative transformName when the original already exists.- Parameters:
transformName- The transformName to find an alternative for- Returns:
- The suggested alternative transformName.
-
getSqlStatements
public List<SqlStatement> getSqlStatements(IVariables variables) throws HopTransformException
Builds a list of all the Sql statements that this pipeline needs in order to work properly.- Parameters:
variables- the variables to resolve variable expressions with- Returns:
- An ArrayList of SqlStatement objects.
- Throws:
HopTransformException- if any errors occur during Sql statement generation
-
getSqlStatements
public List<SqlStatement> getSqlStatements(IVariables variables, IProgressMonitor monitor) throws HopTransformException
Builds a list of all the Sql statements that this pipeline needs in order to work properly.- Parameters:
variables- the variables to resolve variable expressions withmonitor- a progress monitor listener to be updated as the Sql statements are generated- Returns:
- An ArrayList of SqlStatement objects.
- Throws:
HopTransformException- if any errors occur during Sql statement generation
-
getSqlStatementsString
public String getSqlStatementsString(IVariables variables) throws HopTransformException
Get the Sql statements (needed to run this pipeline) as a single String.- Returns:
- the Sql statements needed to run this pipeline
- Throws:
HopTransformException- if any errors occur during Sql statement generation
-
checkTransforms
public void checkTransforms(List<ICheckResult> remarks, boolean onlySelected, IProgressMonitor monitor, IVariables variables, IHopMetadataProvider metadataProvider)
Checks all the transforms and fills a List of (CheckResult) remarks.- Parameters:
remarks- The remarks list to add to.onlySelected- true to check only the selected transforms, false for all transformsmonitor- a progress monitor listener to be updated as the Sql statements are generated
-
getPipelineVersion
public String getPipelineVersion()
Gets the version of the pipeline.- Returns:
- The version of the pipeline
-
setPipelineVersion
public void setPipelineVersion(String pipelineVersion)
Sets the version of the pipeline.- Parameters:
pipelineVersion- The new version description of the pipeline
-
setPipelineStatus
public void setPipelineStatus(int pipelineStatus)
Sets the status of the pipeline.- Parameters:
pipelineStatus- The new status description of the pipeline
-
getPipelineStatus
public int getPipelineStatus()
Gets the status of the pipeline.- Returns:
- The status of the pipeline
-
toString
public String toString()
Gets a textual representation of the pipeline. If its name has been set, it will be returned, otherwise the classname is returned.
-
cancelQueries
public void cancelQueries() throws HopDatabaseExceptionCancels queries opened for checking & fieldprediction.- Throws:
HopDatabaseException- if any errors occur during query cancellation
-
getStringList
public List<StringSearchResult> getStringList(boolean searchTransforms, boolean searchDatabases, boolean searchNotes, boolean includePasswords)
Gets a list of all the strings used in this pipeline. The parameters indicate which collections to search and which to exclude.- Parameters:
searchTransforms- true if transforms should be searched, false otherwisesearchDatabases- true if databases should be searched, false otherwisesearchNotes- true if notes should be searched, false otherwiseincludePasswords- true if passwords should be searched, false otherwise- Returns:
- a list of search results for strings used in the pipeline.
-
getStringList
public List<StringSearchResult> getStringList(boolean searchTransforms, boolean searchDatabases, boolean searchNotes)
Get a list of all the strings used in this pipeline. The parameters indicate which collections to search and which to exclude.- Parameters:
searchTransforms- true if transforms should be searched, false otherwisesearchDatabases- true if databases should be searched, false otherwisesearchNotes- true if notes should be searched, false otherwise- Returns:
- a list of search results for strings used in the pipeline.
-
getUsedVariables
public List<String> getUsedVariables()
Gets a list of the used variables in this pipeline.- Returns:
- a list of the used variables in this pipeline.
-
checkRowMixingStatically
public void checkRowMixingStatically(IVariables variables, TransformMeta transformMeta, IProgressMonitor monitor) throws HopRowException
Check a transform to see if there are no multiple transforms to read from. If so, check to see if the receiving rows are all the same in layout. We only want to ONLY use the DbCache for this to prevent GUI stalls.- Parameters:
transformMeta- the transform to checkmonitor- the monitor- Throws:
HopRowException- in case we detect a row mixing violation
-
setInternalHopVariables
public void setInternalHopVariables(IVariables variables)
Sets the internal hop variables.- Specified by:
setInternalHopVariablesin interfaceIEngineMeta- Specified by:
setInternalHopVariablesin classAbstractMeta- Parameters:
variables- the new internal hop variables
-
setInternalNameHopVariable
protected void setInternalNameHopVariable(IVariables variables)
Sets the internal name hop variable.- Specified by:
setInternalNameHopVariablein classAbstractMeta- Parameters:
variables- the new internal name hop variable
-
setInternalFilenameHopVariables
protected void setInternalFilenameHopVariables(IVariables variables)
Sets the internal filename hop variables.- Specified by:
setInternalFilenameHopVariablesin classAbstractMeta- Parameters:
variables- the new internal filename hop variables
-
setInternalEntryCurrentDirectory
protected void setInternalEntryCurrentDirectory(IVariables variables)
-
findMappingInputTransform
public TransformMeta findMappingInputTransform(String transformName) throws HopTransformException
Finds the mapping input transform with the specified name. If no mapping input transform is found, null is returned- Parameters:
transformName- the name to search for- Returns:
- the transform meta-data corresponding to the desired mapping input transform, or null if no transform was found
- Throws:
HopTransformException- if any errors occur during the search
-
findMappingOutputTransform
public TransformMeta findMappingOutputTransform(String transformName) throws HopTransformException
Finds the mapping output transform with the specified name. If no mapping output transform is found, null is returned.- Parameters:
transformName- the name to search for- Returns:
- the transform meta-data corresponding to the desired mapping input transform, or null if no transform was found
- Throws:
HopTransformException- if any errors occur during the search
-
getResourceDependencies
public List<ResourceReference> getResourceDependencies(IVariables variables)
Gets a list of the resource dependencies.- Returns:
- a list of ResourceReferences
-
exportResources
public String exportResources(IVariables variables, Map<String,ResourceDefinition> definitions, IResourceNaming iResourceNaming, IHopMetadataProvider metadataProvider) throws HopException
Exports the specified objects to a flat-file system, adding content with filename keys to a set of definitions. The supplied resource naming interface allows the object to name appropriately without worrying about those parts of the implementation specific details.- Specified by:
exportResourcesin interfaceIResourceExport- Parameters:
variables- the variable variables to usedefinitions-iResourceNaming-metadataProvider- the metadataProvider in which non-hop metadata could reside.- Returns:
- the filename of the exported resource
- Throws:
HopException- in case something goes wrong during the export
-
isCapturingTransformPerformanceSnapShots
public boolean isCapturingTransformPerformanceSnapShots()
Checks whether the pipeline is capturing transform performance snapshots.- Returns:
- true if the pipeline is capturing transform performance snapshots, false otherwise
-
setCapturingTransformPerformanceSnapShots
public void setCapturingTransformPerformanceSnapShots(boolean capturingTransformPerformanceSnapShots)
Sets whether the pipeline is capturing transform performance snapshots.- Parameters:
capturingTransformPerformanceSnapShots- true if the pipeline is capturing transform performance snapshots, false otherwise
-
getTransformPerformanceCapturingDelay
public long getTransformPerformanceCapturingDelay()
Gets the transform performance capturing delay.- Returns:
- the transform performance capturing delay
-
setTransformPerformanceCapturingDelay
public void setTransformPerformanceCapturingDelay(long transformPerformanceCapturingDelay)
Sets the transform performance capturing delay.- Parameters:
transformPerformanceCapturingDelay- the transformPerformanceCapturingDelay to set
-
getTransformPerformanceCapturingSizeLimit
public String getTransformPerformanceCapturingSizeLimit()
Gets the transform performance capturing size limit.- Returns:
- the transform performance capturing size limit
-
setTransformPerformanceCapturingSizeLimit
public void setTransformPerformanceCapturingSizeLimit(String transformPerformanceCapturingSizeLimit)
Sets the transform performance capturing size limit.- Parameters:
transformPerformanceCapturingSizeLimit- the transform performance capturing size limit to set
-
clearCaches
public void clearCaches()
Clears the transform fields and loop caches.
-
getPipelineType
public PipelineMeta.PipelineType getPipelineType()
Gets the pipeline type.- Returns:
- the pipelineType
-
setPipelineType
public void setPipelineType(PipelineMeta.PipelineType pipelineType)
Sets the pipeline type.- Parameters:
pipelineType- the pipelineType to set
-
addTransformChangeListener
public void addTransformChangeListener(ITransformMetaChangeListener listener)
-
addTransformChangeListener
public void addTransformChangeListener(int p, ITransformMetaChangeListener list)
-
removeTransformChangeListener
public void removeTransformChangeListener(ITransformMetaChangeListener list)
-
notifyAllListeners
public void notifyAllListeners(TransformMeta oldMeta, TransformMeta newMeta)
-
containsTransformMeta
public boolean containsTransformMeta(TransformMeta transformMeta)
-
addMissingPipeline
public void addMissingPipeline(Missing pipeline)
-
removeMissingPipeline
public void removeMissingPipeline(Missing pipeline)
-
hasMissingPlugins
public boolean hasMissingPlugins()
- Overrides:
hasMissingPluginsin classAbstractMeta
-
isEmpty
public boolean isEmpty()
-
setTransforms
public void setTransforms(List<TransformMeta> transforms)
Sets transforms- Parameters:
transforms- value of transforms
-
getHops
public List<PipelineHopMeta> getHops()
Gets hops- Returns:
- value of hops
-
setHops
public void setHops(List<PipelineHopMeta> hops)
Sets hops- Parameters:
hops- value of hops
-
getInfo
public PipelineMetaInfo getInfo()
Gets info- Returns:
- value of info
-
setInfo
public void setInfo(PipelineMetaInfo info)
Sets info- Parameters:
info- value of info
-
getName
public String getName()
Get the name of the pipeline. If the name is synchronized with the filename, we return the base filename.- Specified by:
getNamein interfaceIEngineMeta- Specified by:
getNamein classAbstractMeta- Returns:
- The name of the pipeline
-
setName
public void setName(String newName)
Set the name.- Specified by:
setNamein interfaceIEngineMeta- Specified by:
setNamein classAbstractMeta- Parameters:
newName- The new name
-
isNameSynchronizedWithFilename
public boolean isNameSynchronizedWithFilename()
- Specified by:
isNameSynchronizedWithFilenamein interfaceIEngineMeta- Specified by:
isNameSynchronizedWithFilenamein classAbstractMeta- Returns:
- true if the name is derived from the filename
-
setNameSynchronizedWithFilename
public void setNameSynchronizedWithFilename(boolean nameSynchronizedWithFilename)
- Specified by:
setNameSynchronizedWithFilenamein interfaceIEngineMeta- Specified by:
setNameSynchronizedWithFilenamein classAbstractMeta- Parameters:
nameSynchronizedWithFilename- Set to true if the name is derived from the filename
-
getDescription
public String getDescription()
Gets the description of the workflow.- Specified by:
getDescriptionin classAbstractMeta- Returns:
- The description of the workflow
-
setDescription
public void setDescription(String description)
Set the description of the workflow.- Specified by:
setDescriptionin classAbstractMeta- Parameters:
description- The new description of the workflow
-
getExtendedDescription
public String getExtendedDescription()
Gets the extended description of the workflow.- Specified by:
getExtendedDescriptionin classAbstractMeta- Returns:
- The extended description of the workflow
-
setExtendedDescription
public void setExtendedDescription(String extendedDescription)
Set the description of the workflow.- Specified by:
setExtendedDescriptionin classAbstractMeta- Parameters:
extendedDescription- The new extended description of the workflow
-
getCreatedDate
public Date getCreatedDate()
Gets the date the pipeline was created.- Specified by:
getCreatedDatein interfaceIEngineMeta- Specified by:
getCreatedDatein classAbstractMeta- Returns:
- the date the pipeline was created.
-
setCreatedDate
public void setCreatedDate(Date createdDate)
Sets the date the pipeline was created.- Specified by:
setCreatedDatein interfaceIEngineMeta- Specified by:
setCreatedDatein classAbstractMeta- Parameters:
createdDate- The creation date to set.
-
setCreatedUser
public void setCreatedUser(String createdUser)
Sets the user by whom the pipeline was created.- Specified by:
setCreatedUserin interfaceIEngineMeta- Specified by:
setCreatedUserin classAbstractMeta- Parameters:
createdUser- The user to set.
-
getCreatedUser
public String getCreatedUser()
Gets the user by whom the pipeline was created.- Specified by:
getCreatedUserin interfaceIEngineMeta- Specified by:
getCreatedUserin classAbstractMeta- Returns:
- the user by whom the pipeline was created.
-
setModifiedDate
public void setModifiedDate(Date modifiedDate)
Sets the date the pipeline was modified.- Specified by:
setModifiedDatein interfaceIEngineMeta- Specified by:
setModifiedDatein classAbstractMeta- Parameters:
modifiedDate- The modified date to set.
-
getModifiedDate
public Date getModifiedDate()
Gets the date the pipeline was modified.- Specified by:
getModifiedDatein interfaceIEngineMeta- Specified by:
getModifiedDatein classAbstractMeta- Returns:
- the date the pipeline was modified.
-
setModifiedUser
public void setModifiedUser(String modifiedUser)
Sets the user who last modified the pipeline.- Specified by:
setModifiedUserin interfaceIEngineMeta- Specified by:
setModifiedUserin classAbstractMeta- Parameters:
modifiedUser- The user name to set.
-
getModifiedUser
public String getModifiedUser()
Gets the user who last modified the pipeline.- Specified by:
getModifiedUserin interfaceIEngineMeta- Specified by:
getModifiedUserin classAbstractMeta- Returns:
- the user who last modified the pipeline.
-
-