Class RemotePipelineEngine
- java.lang.Object
-
- org.apache.hop.core.variables.Variables
-
- org.apache.hop.pipeline.engines.remote.RemotePipelineEngine
-
- All Implemented Interfaces:
IExtensionData,ILoggingObject,INamedParameterDefinitions,INamedParameters,IVariables,IPipelineEngine<PipelineMeta>
@PipelineEnginePlugin(id="Remote", name="Hop remote pipeline engine", description="Executes your pipeline on a remote Hop server") public class RemotePipelineEngine extends Variables implements IPipelineEngine<PipelineMeta>
-
-
Field Summary
-
Fields inherited from interface org.apache.hop.pipeline.engine.IPipelineEngine
PIPELINE_IN_PREVIEW_MODE
-
-
Constructor Summary
Constructors Constructor Description RemotePipelineEngine()RemotePipelineEngine(PipelineMeta subject)
-
Method Summary
All Methods Static Methods Instance Methods Concrete Methods Modifier and Type Method Description voidactivateParameters(IVariables variables)voidaddActiveSubPipeline(String subPipelineName, IPipelineEngine subPipeline)Add an active sub-pipeline to allow drill-down in the GUIvoidaddActiveSubWorkflow(String subWorkflowName, IWorkflowEngine<WorkflowMeta> subWorkflow)Add an active sub-workflow to allow drill-down in the GUI<Store extends IExecutionDataSamplerStore,Sampler extends IExecutionDataSampler<Store>>
voidaddExecutionDataSampler(Sampler sampler)Plugins can add individual data samplers which will act upon the rows passing through the transforms in the pipeline.voidaddExecutionFinishedListener(IExecutionFinishedListener executionFinishedListener)Adds a pipeline finished listener.voidaddExecutionStartedListener(IExecutionStartedListener executionStartedListener)Adds a pipeline started listener.voidaddExecutionStoppedListener(IExecutionStoppedListener<IPipelineEngine<PipelineMeta>> listener)Call the given listener lambda when this pipeline engine has stopped execution.voidaddParameterDefinition(String key, String defValue, String description)voidcleanup()This method performs any cleanup operations on the various sub-components of the engine after execution.voidclearParameterValues()voidcopyParametersFromDefinitions(INamedParameterDefinitions definitions)IPipelineEngineRunConfigurationcreateDefaultPipelineEngineRunConfiguration()Ask the engine to generate a default pipeline engine configuration for this enginevoidexecute()Executes the object/subject: calls prepareExecution and startThreads in sequence.IEngineComponentfindComponent(String name, int copyNr)Find a component by name and copy nrIRowSetfindRowSet(String fromTransformName, int fromTransformCopy, String toTransformName, int toTransformCopy)For engines that support it we allow the retrieval of a rowset from one transform copy to anothervoidfirePipelineExecutionFinishedListeners()voidfirePipelineExecutionStartedListeners()voidfirePipelineExecutionStoppedListeners()IPipelineEnginegetActiveSubPipeline(String subPipelineName)Get the active sub-pipeline with the given nameMap<String,IPipelineEngine>getActiveSubPipelines()Gets activeSubPipelinesIWorkflowEngine<WorkflowMeta>getActiveSubWorkflow(String subWorkflowName)Get the active sub-workflow with the given nameMap<String,IWorkflowEngine<WorkflowMeta>>getActiveSubWorkflows()Gets activeSubWorkflowsList<IEngineComponent>getComponentCopies(String name)Find all component copies by nameStringgetComponentLogText(String componentName, int copyNr)Retrieve the logging text of a particular component in the engineList<IEngineComponent>getComponents()Get a list of componentsStringgetContainerId()Gets containerIdPipelineEngineCapabilitiesgetEngineCapabilities()Gets engineCapabilitiesEngineMetricsgetEngineMetrics()Gets engineMetricsEngineMetricsgetEngineMetrics(String componentName, int copyNr)Get the engine metrics for a specific component name and/or copy nrintgetErrors()Gets errorsDategetExecutionEndDate()List<IExecutionFinishedListener<IPipelineEngine<PipelineMeta>>>getExecutionFinishedListeners()Gets executionFinishedListenersDategetExecutionStartDate()List<IExecutionStartedListener<IPipelineEngine<PipelineMeta>>>getExecutionStartedListeners()Gets executionStartedListenersList<IExecutionStoppedListener<IPipelineEngine<PipelineMeta>>>getExecutionStoppedListeners()Gets executionStoppedListenersMap<String,Object>getExtensionDataMap()Gets extensionDataMapintgetFeedbackSize()Gets feedbackSizeStringgetFilename()HopServergetHopServer()Gets hopServerintgetLastLogLineNr()Gets lastLogLineNrILogChannelgetLogChannel()Gets logChannelStringgetLogChannelId()The log channel ID if there is anyILoggingObjectgetLoggingObject()Gets loggingObjectLogLevelgetLogLevel()Gets logLevelIHopMetadataProvidergetMetadataProvider()Gets metadataProviderINamedParametersgetNamedParams()Gets namedParamsStringgetObjectCopy()StringgetObjectName()LoggingObjectTypegetObjectType()StringgetParameterDefault(String key)StringgetParameterDescription(String key)StringgetParameterValue(String key)ILoggingObjectgetParent()Gets parentIPipelineEnginegetParentPipeline()Gets parentPipelineIWorkflowEngine<WorkflowMeta>getParentWorkflow()Gets parentWorkflowPipelineMetagetPipelineMeta()Gets subjectPipelineRunConfigurationgetPipelineRunConfiguration()Gets pipelineRunConfigurationStringgetPluginId()Gets pluginIdResultgetPreviousResult()Gets previousResultDategetRegistrationDate()ResultgetResult()Get the result of the execution after it's done, a resumelonggetServerPollDelay()Gets serverPollDelaylonggetServerPollInterval()Gets serverPollIntervalEngineComponent.ComponentExecutionStatusgetStatus()Gets statusStringgetStatusDescription()booleanhasHaltedComponents()See if there are any halted components in the engine: actions, auditing, ...booleanisFeedbackShown()Gets feedbackShownbooleanisFinished()Gets finishedbooleanisForcingSeparateLogging()booleanisGatheringMetrics()booleanisHasHaltedComponents()Gets hasHaltedComponentsbooleanisPaused()Gets pausedbooleanisPreparing()Gets preparingbooleanisPreview()Gets previewbooleanisReadyToStart()Gets readyToStartbooleanisRunning()Gets runningbooleanisSafeModeEnabled()True if the engine is doing extra validations at runtime to detect possible issues with data types and so on.booleanisStopped()Gets stoppedstatic booleanisVariablePassedToRemoteServer(String variableName)String[]listParameters()voidpauseExecution()Pauses the execution (all components).voidpipelineCompleted()This method is called when all the finished listeners are executed and when the pipeline has completed.voidprepareExecution()Prepares for execution.voidremoveAllParameters()voidresumeExecution()Resume the execution (all components).voidretrieveComponentOutput(IVariables variables, String componentName, int copyNr, int nrRows, IPipelineComponentRowsReceived rowsReceived)Retrieve output rows from a component copy.voidsetActiveSubPipelines(Map<String,IPipelineEngine> activeSubPipelines)voidsetActiveSubWorkflows(Map<String,IWorkflowEngine<WorkflowMeta>> activeSubWorkflows)voidsetContainerId(String containerId)The unique ID this pipeline engine has while executing in a container like a web-server and so on.voidsetEngineMetrics(EngineMetrics engineMetrics)voidsetErrors(int errors)voidsetExecutionFinishedListeners(List<IExecutionFinishedListener<IPipelineEngine<PipelineMeta>>> executionFinishedListeners)voidsetExecutionStartedListeners(List<IExecutionStartedListener<IPipelineEngine<PipelineMeta>>> executionStartedListeners)voidsetExecutionStoppedListeners(List<IExecutionStoppedListener<IPipelineEngine<PipelineMeta>>> executionStoppedListeners)voidsetExtensionDataMap(Map<String,Object> extensionDataMap)voidsetFeedbackShown(boolean feedbackShown)voidsetFeedbackSize(int feedbackSize)voidsetFinished(boolean finished)voidsetForcingSeparateLogging(boolean forcingSeparateLogging)voidsetGatheringMetrics(boolean gatheringMetrics)voidsetHasHaltedComponents(boolean hasHaltedComponents)voidsetHopServer(HopServer hopServer)voidsetInternalHopVariables(IVariables var)Make the engine define internal hop variables based on the given variables, the filename and so onvoidsetLastLogLineNr(int lastLogLineNr)voidsetLogChannel(ILogChannel log)Sets the log channel.voidsetLoggingObject(ILoggingObject loggingObject)voidsetLogLevel(LogLevel logLevel)voidsetMetadataProvider(IHopMetadataProvider metadataProvider)voidsetNamedParams(INamedParameters namedParams)voidsetParameterValue(String key, String value)voidsetParent(ILoggingObject parent)Sets the parent logging object.voidsetParentPipeline(IPipelineEngine parentPipeline)Inform the pipeline that it's being executed as part of another pipelinevoidsetParentWorkflow(IWorkflowEngine<WorkflowMeta> parentWorkflow)Inform the pipeline that it's being executed as part of a workflowvoidsetPaused(boolean paused)voidsetPipelineMeta(PipelineMeta subject)voidsetPipelineRunConfiguration(PipelineRunConfiguration pipelineRunConfiguration)voidsetPluginId(String pluginId)voidsetPreparing(boolean preparing)voidsetPreview(boolean preview)Temporary until we find a better way to preview/debugvoidsetPreviousResult(Result previousResult)Inform the pipeline about a previous execution result in a workflow or pipelinevoidsetReadyToStart(boolean readyToStart)voidsetRunning(boolean running)voidsetServerPollDelay(long serverPollDelay)voidsetServerPollInterval(long serverPollInterval)voidsetStatus(EngineComponent.ComponentExecutionStatus status)voidsetStatusDescription(String statusDescription)voidsetStopped(boolean stopped)voidstartThreads()Starts the engine itself after prepareExecution.voidstopAll()Stop the pipeline on the servervoidwaitUntilFinished()Wait until the execution is done.-
Methods inherited from class org.apache.hop.core.variables.Variables
copyFrom, getADefaultVariableSpace, getParentVariables, getVariable, getVariable, getVariableBoolean, getVariableNames, initializeFrom, resolve, resolve, resolve, setParentVariables, setVariable, setVariables, shareWith
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface org.apache.hop.core.variables.IVariables
copyFrom, getParentVariables, getVariable, getVariable, getVariableBoolean, getVariableNames, initializeFrom, resolve, resolve, resolve, setParentVariables, setVariable, setVariables, shareWith
-
-
-
-
Field Detail
-
CONFIGURATION_IN_EXPORT_FILENAME
public static final String CONFIGURATION_IN_EXPORT_FILENAME
Constant specifying a filename containing XML to inject into a ZIP file created during resource export.- See Also:
- Constant Field Values
-
subject
protected PipelineMeta subject
-
pluginId
protected String pluginId
-
pipelineRunConfiguration
protected PipelineRunConfiguration pipelineRunConfiguration
-
remotePipelineRunConfiguration
protected RemotePipelineRunConfiguration remotePipelineRunConfiguration
-
preparing
protected boolean preparing
-
readyToStart
protected boolean readyToStart
-
running
protected boolean running
-
finished
protected boolean finished
-
stopped
protected boolean stopped
-
paused
protected boolean paused
-
hasHaltedComponents
protected boolean hasHaltedComponents
-
preview
protected boolean preview
-
errors
protected int errors
-
metadataProvider
protected IHopMetadataProvider metadataProvider
-
logChannel
protected ILogChannel logChannel
-
loggingObject
protected ILoggingObject loggingObject
-
engineMetrics
protected EngineMetrics engineMetrics
-
previousResult
protected Result previousResult
-
hopServer
protected HopServer hopServer
-
parent
protected ILoggingObject parent
-
parentPipeline
protected IPipelineEngine parentPipeline
-
parentWorkflow
protected IWorkflowEngine<WorkflowMeta> parentWorkflow
-
logLevel
protected LogLevel logLevel
-
feedbackShown
protected boolean feedbackShown
-
feedbackSize
protected int feedbackSize
-
containerId
protected String containerId
-
executionStartedListeners
protected List<IExecutionStartedListener<IPipelineEngine<PipelineMeta>>> executionStartedListeners
A list of started listeners attached to the pipeline.
-
executionFinishedListeners
protected List<IExecutionFinishedListener<IPipelineEngine<PipelineMeta>>> executionFinishedListeners
A list of finished listeners attached to the pipeline.
-
executionStoppedListeners
protected List<IExecutionStoppedListener<IPipelineEngine<PipelineMeta>>> executionStoppedListeners
A list of stop-event listeners attached to the pipeline.
-
activeSubPipelines
protected Map<String,IPipelineEngine> activeSubPipelines
The active sub-pipelines.
-
activeSubWorkflows
protected Map<String,IWorkflowEngine<WorkflowMeta>> activeSubWorkflows
The active subjobs
-
lastLogLineNr
protected int lastLogLineNr
-
refreshTimer
protected Timer refreshTimer
-
namedParams
protected INamedParameters namedParams
The named parameters.
-
serverPollDelay
protected long serverPollDelay
-
serverPollInterval
protected long serverPollInterval
-
-
Constructor Detail
-
RemotePipelineEngine
public RemotePipelineEngine()
-
RemotePipelineEngine
public RemotePipelineEngine(PipelineMeta subject)
-
-
Method Detail
-
createDefaultPipelineEngineRunConfiguration
public IPipelineEngineRunConfiguration createDefaultPipelineEngineRunConfiguration()
Description copied from interface:IPipelineEngineAsk the engine to generate a default pipeline engine configuration for this engine- Specified by:
createDefaultPipelineEngineRunConfigurationin interfaceIPipelineEngine<PipelineMeta>- Returns:
- a new pipeline engine run configuration
-
prepareExecution
public void prepareExecution() throws HopExceptionDescription copied from interface:IPipelineEnginePrepares for execution. This includes setting the parameters as well as preparing anything needed in terms of resources, threads,.... It runs initialization on all the work that is needed to be done up-front.- Specified by:
prepareExecutionin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException- In case of an initialization error
-
isVariablePassedToRemoteServer
public static boolean isVariablePassedToRemoteServer(String variableName)
-
startThreads
public void startThreads() throws HopExceptionDescription copied from interface:IPipelineEngineStarts the engine itself after prepareExecution.- Specified by:
startThreadsin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException- If there is an engine error during execution.
-
getStatusDescription
public String getStatusDescription()
- Specified by:
getStatusDescriptionin interfaceIPipelineEngine<PipelineMeta>- Returns:
- Get a status description of the state of the engine (running, stopped, finished, paused, halted, ...)
-
execute
public void execute() throws HopExceptionDescription copied from interface:IPipelineEngineExecutes the object/subject: calls prepareExecution and startThreads in sequence.- Specified by:
executein interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException- if the object execution could not be prepared (initialized)
-
getEngineMetrics
public EngineMetrics getEngineMetrics(String componentName, int copyNr)
Description copied from interface:IPipelineEngineGet the engine metrics for a specific component name and/or copy nr- Specified by:
getEngineMetricsin interfaceIPipelineEngine<PipelineMeta>- Parameters:
componentName- the name of the component or null for all componentscopyNr- The copy nr to select or a negative number for all components- Returns:
- The engine metrics for the given
-
cleanup
public void cleanup()
Description copied from interface:IPipelineEngineThis method performs any cleanup operations on the various sub-components of the engine after execution.- Specified by:
cleanupin interfaceIPipelineEngine<PipelineMeta>
-
waitUntilFinished
public void waitUntilFinished()
Description copied from interface:IPipelineEngineWait until the execution is done.- Specified by:
waitUntilFinishedin interfaceIPipelineEngine<PipelineMeta>
-
stopAll
public void stopAll()
Stop the pipeline on the server- Specified by:
stopAllin interfaceIPipelineEngine<PipelineMeta>
-
hasHaltedComponents
public boolean hasHaltedComponents()
Description copied from interface:IPipelineEngineSee if there are any halted components in the engine: actions, auditing, ...- Specified by:
hasHaltedComponentsin interfaceIPipelineEngine<PipelineMeta>- Returns:
- True if there are halted components
-
pauseExecution
public void pauseExecution()
Description copied from interface:IPipelineEnginePauses the execution (all components).- Specified by:
pauseExecutionin interfaceIPipelineEngine<PipelineMeta>
-
resumeExecution
public void resumeExecution()
Description copied from interface:IPipelineEngineResume the execution (all components).- Specified by:
resumeExecutionin interfaceIPipelineEngine<PipelineMeta>
-
addExecutionStartedListener
public void addExecutionStartedListener(IExecutionStartedListener executionStartedListener)
Adds a pipeline started listener.- Specified by:
addExecutionStartedListenerin interfaceIPipelineEngine<PipelineMeta>- Parameters:
executionStartedListener- the pipeline started listener
-
addExecutionFinishedListener
public void addExecutionFinishedListener(IExecutionFinishedListener executionFinishedListener)
Adds a pipeline finished listener.- Specified by:
addExecutionFinishedListenerin interfaceIPipelineEngine<PipelineMeta>- Parameters:
executionFinishedListener- the pipeline finished listener
-
pipelineCompleted
public void pipelineCompleted() throws HopExceptionDescription copied from interface:IPipelineEngineThis method is called when all the finished listeners are executed and when the pipeline has completed.- Specified by:
pipelineCompletedin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
getComponentLogText
public String getComponentLogText(String componentName, int copyNr)
Description copied from interface:IPipelineEngineRetrieve the logging text of a particular component in the engine- Specified by:
getComponentLogTextin interfaceIPipelineEngine<PipelineMeta>- Parameters:
componentName- The name of the component (transform)copyNr- The copy number if multiple components are running in parallel (0 based).- Returns:
- The logging text
-
getComponents
public List<IEngineComponent> getComponents()
Description copied from interface:IPipelineEngineGet a list of components- Specified by:
getComponentsin interfaceIPipelineEngine<PipelineMeta>- Returns:
- The list of components in the engine
-
getComponentCopies
public List<IEngineComponent> getComponentCopies(String name)
Description copied from interface:IPipelineEngineFind all component copies by name- Specified by:
getComponentCopiesin interfaceIPipelineEngine<PipelineMeta>- Parameters:
name- The name of the component to look for (transform)- Returns:
- The list of components found
-
findComponent
public IEngineComponent findComponent(String name, int copyNr)
Description copied from interface:IPipelineEngineFind a component by name and copy nr- Specified by:
findComponentin interfaceIPipelineEngine<PipelineMeta>- Parameters:
name- The name of the component to look for (transform)copyNr- The copy number to match- Returns:
- The component or null if nothing was found
-
getResult
public Result getResult()
Description copied from interface:IPipelineEngineGet the result of the execution after it's done, a resume- Specified by:
getResultin interfaceIPipelineEngine<PipelineMeta>- Returns:
- The Result of the execution of the object
-
retrieveComponentOutput
public void retrieveComponentOutput(IVariables variables, String componentName, int copyNr, int nrRows, IPipelineComponentRowsReceived rowsReceived) throws HopException
Description copied from interface:IPipelineEngineRetrieve output rows from a component copy. Pass the rows to the rows received lambda.- Specified by:
retrieveComponentOutputin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
getExecutionStartDate
public Date getExecutionStartDate()
- Specified by:
getExecutionStartDatein interfaceIPipelineEngine<PipelineMeta>- Returns:
- The start date of the pipeline execution
-
getExecutionEndDate
public Date getExecutionEndDate()
- Specified by:
getExecutionEndDatein interfaceIPipelineEngine<PipelineMeta>- Returns:
- The end date of the pipeline preparation
-
isSafeModeEnabled
public boolean isSafeModeEnabled()
Description copied from interface:IPipelineEngineTrue if the engine is doing extra validations at runtime to detect possible issues with data types and so on.- Specified by:
isSafeModeEnabledin interfaceIPipelineEngine<PipelineMeta>- Returns:
- True if safe mode is enabled.
-
findRowSet
public IRowSet findRowSet(String fromTransformName, int fromTransformCopy, String toTransformName, int toTransformCopy)
Description copied from interface:IPipelineEngineFor engines that support it we allow the retrieval of a rowset from one transform copy to another- Specified by:
findRowSetin interfaceIPipelineEngine<PipelineMeta>- Returns:
- The rowset if one was found.
-
getObjectName
public String getObjectName()
- Specified by:
getObjectNamein interfaceILoggingObject
-
getFilename
public String getFilename()
- Specified by:
getFilenamein interfaceILoggingObject
-
getObjectType
public LoggingObjectType getObjectType()
- Specified by:
getObjectTypein interfaceILoggingObject
-
getObjectCopy
public String getObjectCopy()
- Specified by:
getObjectCopyin interfaceILoggingObject
-
getRegistrationDate
public Date getRegistrationDate()
- Specified by:
getRegistrationDatein interfaceILoggingObject
-
isGatheringMetrics
public boolean isGatheringMetrics()
- Specified by:
isGatheringMetricsin interfaceILoggingObject
-
setGatheringMetrics
public void setGatheringMetrics(boolean gatheringMetrics)
- Specified by:
setGatheringMetricsin interfaceILoggingObject
-
setForcingSeparateLogging
public void setForcingSeparateLogging(boolean forcingSeparateLogging)
- Specified by:
setForcingSeparateLoggingin interfaceILoggingObject
-
isForcingSeparateLogging
public boolean isForcingSeparateLogging()
- Specified by:
isForcingSeparateLoggingin interfaceILoggingObject
-
getLogChannelId
public String getLogChannelId()
Description copied from interface:IPipelineEngineThe log channel ID if there is any- Specified by:
getLogChannelIdin interfaceILoggingObject- Specified by:
getLogChannelIdin interfaceIPipelineEngine<PipelineMeta>
-
addActiveSubPipeline
public void addActiveSubPipeline(String subPipelineName, IPipelineEngine subPipeline)
Description copied from interface:IPipelineEngineAdd an active sub-pipeline to allow drill-down in the GUI- Specified by:
addActiveSubPipelinein interfaceIPipelineEngine<PipelineMeta>
-
getActiveSubPipeline
public IPipelineEngine getActiveSubPipeline(String subPipelineName)
Description copied from interface:IPipelineEngineGet the active sub-pipeline with the given name- Specified by:
getActiveSubPipelinein interfaceIPipelineEngine<PipelineMeta>- Returns:
- The active pipeline engine or null if it was not found
-
addActiveSubWorkflow
public void addActiveSubWorkflow(String subWorkflowName, IWorkflowEngine<WorkflowMeta> subWorkflow)
Description copied from interface:IPipelineEngineAdd an active sub-workflow to allow drill-down in the GUI- Specified by:
addActiveSubWorkflowin interfaceIPipelineEngine<PipelineMeta>
-
getActiveSubWorkflow
public IWorkflowEngine<WorkflowMeta> getActiveSubWorkflow(String subWorkflowName)
Description copied from interface:IPipelineEngineGet the active sub-workflow with the given name- Specified by:
getActiveSubWorkflowin interfaceIPipelineEngine<PipelineMeta>- Returns:
- The active workflow or null if nothing was found
-
setInternalHopVariables
public void setInternalHopVariables(IVariables var)
Description copied from interface:IPipelineEngineMake the engine define internal hop variables based on the given variables, the filename and so on- Specified by:
setInternalHopVariablesin interfaceIPipelineEngine<PipelineMeta>- Parameters:
var- TODO get rid of this method, internal variables should be available
-
getParentPipeline
public IPipelineEngine getParentPipeline()
Gets parentPipeline- Specified by:
getParentPipelinein interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of parentPipeline
-
setParentPipeline
public void setParentPipeline(IPipelineEngine parentPipeline)
Description copied from interface:IPipelineEngineInform the pipeline that it's being executed as part of another pipeline- Specified by:
setParentPipelinein interfaceIPipelineEngine<PipelineMeta>- Parameters:
parentPipeline- The parentPipeline to set
-
getParentWorkflow
public IWorkflowEngine<WorkflowMeta> getParentWorkflow()
Gets parentWorkflow- Specified by:
getParentWorkflowin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of parentWorkflow
-
setParentWorkflow
public void setParentWorkflow(IWorkflowEngine<WorkflowMeta> parentWorkflow)
Description copied from interface:IPipelineEngineInform the pipeline that it's being executed as part of a workflow- Specified by:
setParentWorkflowin interfaceIPipelineEngine<PipelineMeta>- Parameters:
parentWorkflow- The parentWorkflow to set
-
getExecutionStartedListeners
public List<IExecutionStartedListener<IPipelineEngine<PipelineMeta>>> getExecutionStartedListeners()
Gets executionStartedListeners- Returns:
- value of executionStartedListeners
-
setExecutionStartedListeners
public void setExecutionStartedListeners(List<IExecutionStartedListener<IPipelineEngine<PipelineMeta>>> executionStartedListeners)
- Parameters:
executionStartedListeners- The executionStartedListeners to set
-
getExecutionFinishedListeners
public List<IExecutionFinishedListener<IPipelineEngine<PipelineMeta>>> getExecutionFinishedListeners()
Gets executionFinishedListeners- Returns:
- value of executionFinishedListeners
-
setExecutionFinishedListeners
public void setExecutionFinishedListeners(List<IExecutionFinishedListener<IPipelineEngine<PipelineMeta>>> executionFinishedListeners)
- Parameters:
executionFinishedListeners- The executionFinishedListeners to set
-
addExecutionStoppedListener
public void addExecutionStoppedListener(IExecutionStoppedListener<IPipelineEngine<PipelineMeta>> listener) throws HopException
Description copied from interface:IPipelineEngineCall the given listener lambda when this pipeline engine has stopped execution.- Specified by:
addExecutionStoppedListenerin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
firePipelineExecutionStartedListeners
public void firePipelineExecutionStartedListeners() throws HopException- Specified by:
firePipelineExecutionStartedListenersin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
firePipelineExecutionFinishedListeners
public void firePipelineExecutionFinishedListeners() throws HopException- Specified by:
firePipelineExecutionFinishedListenersin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
firePipelineExecutionStoppedListeners
public void firePipelineExecutionStoppedListeners() throws HopException- Specified by:
firePipelineExecutionStoppedListenersin interfaceIPipelineEngine<PipelineMeta>- Throws:
HopException
-
getExecutionStoppedListeners
public List<IExecutionStoppedListener<IPipelineEngine<PipelineMeta>>> getExecutionStoppedListeners()
Gets executionStoppedListeners- Returns:
- value of executionStoppedListeners
-
setExecutionStoppedListeners
public void setExecutionStoppedListeners(List<IExecutionStoppedListener<IPipelineEngine<PipelineMeta>>> executionStoppedListeners)
- Parameters:
executionStoppedListeners- The executionStoppedListeners to set
-
addExecutionDataSampler
public <Store extends IExecutionDataSamplerStore,Sampler extends IExecutionDataSampler<Store>> void addExecutionDataSampler(Sampler sampler) throws HopException
Description copied from interface:IPipelineEnginePlugins can add individual data samplers which will act upon the rows passing through the transforms in the pipeline. The execution information collected at this level will be sent to the execution information location if any is defined in the run configuration used to execute this pipeline.- Specified by:
addExecutionDataSamplerin interfaceIPipelineEngine<PipelineMeta>- Parameters:
sampler- The sampler to use- Throws:
HopException
-
getErrors
public int getErrors()
Gets errors- Specified by:
getErrorsin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of errors
-
setErrors
public void setErrors(int errors)
- Parameters:
errors- The errors to set
-
getPipelineRunConfiguration
public PipelineRunConfiguration getPipelineRunConfiguration()
Gets pipelineRunConfiguration- Specified by:
getPipelineRunConfigurationin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of pipelineRunConfiguration
-
setPipelineRunConfiguration
public void setPipelineRunConfiguration(PipelineRunConfiguration pipelineRunConfiguration)
- Specified by:
setPipelineRunConfigurationin interfaceIPipelineEngine<PipelineMeta>- Parameters:
pipelineRunConfiguration- The pipelineRunConfiguration to set
-
getPipelineMeta
public PipelineMeta getPipelineMeta()
Gets subject- Specified by:
getPipelineMetain interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of subject
-
setPipelineMeta
public void setPipelineMeta(PipelineMeta subject)
- Specified by:
setPipelineMetain interfaceIPipelineEngine<PipelineMeta>- Parameters:
subject- The subject to set
-
getPluginId
public String getPluginId()
Gets pluginId- Specified by:
getPluginIdin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of pluginId
-
setPluginId
public void setPluginId(String pluginId)
- Specified by:
setPluginIdin interfaceIPipelineEngine<PipelineMeta>- Parameters:
pluginId- The pluginId to set
-
isPreparing
public boolean isPreparing()
Gets preparing- Specified by:
isPreparingin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of preparing
-
setPreparing
public void setPreparing(boolean preparing)
- Parameters:
preparing- The preparing to set
-
isReadyToStart
public boolean isReadyToStart()
Gets readyToStart- Specified by:
isReadyToStartin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of readyToStart
-
setReadyToStart
public void setReadyToStart(boolean readyToStart)
- Parameters:
readyToStart- The readyToStart to set
-
isRunning
public boolean isRunning()
Gets running- Specified by:
isRunningin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of running
-
setRunning
public void setRunning(boolean running)
- Parameters:
running- The running to set
-
isStopped
public boolean isStopped()
Gets stopped- Specified by:
isStoppedin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of stopped
-
setStopped
public void setStopped(boolean stopped)
- Parameters:
stopped- The stopped to set
-
getMetadataProvider
public IHopMetadataProvider getMetadataProvider()
Gets metadataProvider- Specified by:
getMetadataProviderin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of metadataProvider
-
setMetadataProvider
public void setMetadataProvider(IHopMetadataProvider metadataProvider)
- Specified by:
setMetadataProviderin interfaceIPipelineEngine<PipelineMeta>- Parameters:
metadataProvider- The metadataProvider to set
-
getLogChannel
public ILogChannel getLogChannel()
Gets logChannel- Specified by:
getLogChannelin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of logChannel
-
setLogChannel
public void setLogChannel(ILogChannel log)
Description copied from interface:IPipelineEngineSets the log channel. For testing only- Specified by:
setLogChannelin interfaceIPipelineEngine<PipelineMeta>- Parameters:
log- The logChannel to set
-
getHopServer
public HopServer getHopServer()
Gets hopServer- Returns:
- value of hopServer
-
setHopServer
public void setHopServer(HopServer hopServer)
- Parameters:
hopServer- The hopServer to set
-
getEngineMetrics
public EngineMetrics getEngineMetrics()
Gets engineMetrics- Specified by:
getEngineMetricsin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of engineMetrics
-
setEngineMetrics
public void setEngineMetrics(EngineMetrics engineMetrics)
- Parameters:
engineMetrics- The engineMetrics to set
-
isFinished
public boolean isFinished()
Gets finished- Specified by:
isFinishedin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of finished
-
setFinished
public void setFinished(boolean finished)
- Parameters:
finished- The finished to set
-
isPaused
public boolean isPaused()
Gets paused- Specified by:
isPausedin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of paused
-
setPaused
public void setPaused(boolean paused)
- Parameters:
paused- The paused to set
-
getParent
public ILoggingObject getParent()
Gets parent- Specified by:
getParentin interfaceILoggingObject- Returns:
- value of parent
-
setParent
public void setParent(ILoggingObject parent)
Description copied from interface:IPipelineEngineSets the parent logging object.- Specified by:
setParentin interfaceIPipelineEngine<PipelineMeta>- Parameters:
parent- The parent to set
-
getLogLevel
public LogLevel getLogLevel()
Gets logLevel- Specified by:
getLogLevelin interfaceILoggingObject- Specified by:
getLogLevelin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of logLevel
-
setLogLevel
public void setLogLevel(LogLevel logLevel)
- Specified by:
setLogLevelin interfaceIPipelineEngine<PipelineMeta>- Parameters:
logLevel- The logLevel to set
-
isPreview
public boolean isPreview()
Gets preview- Specified by:
isPreviewin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of preview
-
setPreview
public void setPreview(boolean preview)
Description copied from interface:IPipelineEngineTemporary until we find a better way to preview/debug- Specified by:
setPreviewin interfaceIPipelineEngine<PipelineMeta>- Parameters:
preview- The preview to set
-
isHasHaltedComponents
public boolean isHasHaltedComponents()
Gets hasHaltedComponents- Returns:
- value of hasHaltedComponents
-
setHasHaltedComponents
public void setHasHaltedComponents(boolean hasHaltedComponents)
- Parameters:
hasHaltedComponents- The hasHaltedComponents to set
-
isFeedbackShown
public boolean isFeedbackShown()
Gets feedbackShown- Specified by:
isFeedbackShownin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of feedbackShown
-
setFeedbackShown
public void setFeedbackShown(boolean feedbackShown)
- Parameters:
feedbackShown- The feedbackShown to set
-
getFeedbackSize
public int getFeedbackSize()
Gets feedbackSize- Specified by:
getFeedbackSizein interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of feedbackSize
-
setFeedbackSize
public void setFeedbackSize(int feedbackSize)
- Parameters:
feedbackSize- The feedbackSize to set
-
getLastLogLineNr
public int getLastLogLineNr()
Gets lastLogLineNr- Returns:
- value of lastLogLineNr
-
setLastLogLineNr
public void setLastLogLineNr(int lastLogLineNr)
- Parameters:
lastLogLineNr- The lastLogLineNr to set
-
getLoggingObject
public ILoggingObject getLoggingObject()
Gets loggingObject- Returns:
- value of loggingObject
-
setLoggingObject
public void setLoggingObject(ILoggingObject loggingObject)
- Parameters:
loggingObject- The loggingObject to set
-
getPreviousResult
public Result getPreviousResult()
Gets previousResult- Specified by:
getPreviousResultin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of previousResult
-
setPreviousResult
public void setPreviousResult(Result previousResult)
Description copied from interface:IPipelineEngineInform the pipeline about a previous execution result in a workflow or pipeline- Specified by:
setPreviousResultin interfaceIPipelineEngine<PipelineMeta>- Parameters:
previousResult- The previousResult to set
-
getActiveSubPipelines
public Map<String,IPipelineEngine> getActiveSubPipelines()
Gets activeSubPipelines- Returns:
- value of activeSubPipelines
-
setActiveSubPipelines
public void setActiveSubPipelines(Map<String,IPipelineEngine> activeSubPipelines)
- Parameters:
activeSubPipelines- The activeSubPipelines to set
-
getActiveSubWorkflows
public Map<String,IWorkflowEngine<WorkflowMeta>> getActiveSubWorkflows()
Gets activeSubWorkflows- Returns:
- value of activeSubWorkflows
-
setActiveSubWorkflows
public void setActiveSubWorkflows(Map<String,IWorkflowEngine<WorkflowMeta>> activeSubWorkflows)
- Parameters:
activeSubWorkflows- The activeSubWorkflows to set
-
addParameterDefinition
public void addParameterDefinition(String key, String defValue, String description) throws DuplicateParamException
- Specified by:
addParameterDefinitionin interfaceINamedParameterDefinitions- Throws:
DuplicateParamException
-
getParameterDescription
public String getParameterDescription(String key) throws UnknownParamException
- Specified by:
getParameterDescriptionin interfaceINamedParameterDefinitions- Throws:
UnknownParamException
-
getParameterDefault
public String getParameterDefault(String key) throws UnknownParamException
- Specified by:
getParameterDefaultin interfaceINamedParameterDefinitions- Throws:
UnknownParamException
-
getParameterValue
public String getParameterValue(String key) throws UnknownParamException
- Specified by:
getParameterValuein interfaceINamedParameters- Throws:
UnknownParamException
-
listParameters
public String[] listParameters()
- Specified by:
listParametersin interfaceINamedParameterDefinitions
-
setParameterValue
public void setParameterValue(String key, String value) throws UnknownParamException
- Specified by:
setParameterValuein interfaceINamedParameters- Throws:
UnknownParamException
-
removeAllParameters
public void removeAllParameters()
- Specified by:
removeAllParametersin interfaceINamedParameterDefinitions- Specified by:
removeAllParametersin interfaceINamedParameters
-
clearParameterValues
public void clearParameterValues()
- Specified by:
clearParameterValuesin interfaceINamedParameters
-
activateParameters
public void activateParameters(IVariables variables)
- Specified by:
activateParametersin interfaceINamedParameters
-
copyParametersFromDefinitions
public void copyParametersFromDefinitions(INamedParameterDefinitions definitions)
- Specified by:
copyParametersFromDefinitionsin interfaceINamedParameters
-
getEngineCapabilities
public PipelineEngineCapabilities getEngineCapabilities()
Gets engineCapabilities- Specified by:
getEngineCapabilitiesin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of engineCapabilities
-
getNamedParams
public INamedParameters getNamedParams()
Gets namedParams- Returns:
- value of namedParams
-
setNamedParams
public void setNamedParams(INamedParameters namedParams)
- Parameters:
namedParams- The namedParams to set
-
getContainerId
public String getContainerId()
Gets containerId- Specified by:
getContainerIdin interfaceILoggingObject- Returns:
- value of containerId
-
setContainerId
public void setContainerId(String containerId)
Description copied from interface:IPipelineEngineThe unique ID this pipeline engine has while executing in a container like a web-server and so on. When we have multiple pipelines running with the same name- Specified by:
setContainerIdin interfaceIPipelineEngine<PipelineMeta>- Parameters:
containerId- The containerId to set
-
getStatus
public EngineComponent.ComponentExecutionStatus getStatus()
Gets status- Returns:
- value of status
-
getExtensionDataMap
public Map<String,Object> getExtensionDataMap()
Gets extensionDataMap- Specified by:
getExtensionDataMapin interfaceIExtensionData- Specified by:
getExtensionDataMapin interfaceIPipelineEngine<PipelineMeta>- Returns:
- value of extensionDataMap
-
setStatusDescription
public void setStatusDescription(String statusDescription)
- Parameters:
statusDescription- The statusDescription to set
-
setStatus
public void setStatus(EngineComponent.ComponentExecutionStatus status)
- Parameters:
status- The status to set
-
getServerPollDelay
public long getServerPollDelay()
Gets serverPollDelay- Returns:
- value of serverPollDelay
-
setServerPollDelay
public void setServerPollDelay(long serverPollDelay)
- Parameters:
serverPollDelay- The serverPollDelay to set
-
getServerPollInterval
public long getServerPollInterval()
Gets serverPollInterval- Returns:
- value of serverPollInterval
-
setServerPollInterval
public void setServerPollInterval(long serverPollInterval)
- Parameters:
serverPollInterval- The serverPollInterval to set
-
-