Package org.apache.hop.pipeline.engines.local

Class LocalPipelineEngine

java.lang.Object

org.apache.hop.pipeline.Pipeline

org.apache.hop.pipeline.engines.local.LocalPipelineEngine

All Implemented Interfaces:

IExecutor, IExtensionData, IHasLogChannel, ILoggingObject, INamedParameterDefinitions, INamedParameters, IVariables, IPipelineEngine<PipelineMeta>

@PipelineEnginePlugin(id="Local",
                      name="Hop local pipeline engine",
                      description="Executes your pipeline locally in a multi-threaded fashion")
public class LocalPipelineEngine
extends Pipeline
implements IPipelineEngine<PipelineMeta>

Field Summary

Fields inherited from class org.apache.hop.pipeline.Pipeline

arguments, CONFIGURATION_IN_EXPORT_FILENAME, containerObjectId, dataSamplers, feedbackShown, feedbackSize, log, logCommitSize, logLevel, metadataProvider, METRIC_BUFFER_IN, METRIC_BUFFER_OUT, METRIC_ERROR, METRIC_FLUSH_BUFFER, METRIC_INIT, METRIC_INPUT, METRIC_NAME_BUFFER_IN, METRIC_NAME_BUFFER_OUT, METRIC_NAME_ERROR, METRIC_NAME_FLUSH_BUFFER, METRIC_NAME_INIT, METRIC_NAME_INPUT, METRIC_NAME_OUTPUT, METRIC_NAME_READ, METRIC_NAME_REJECTED, METRIC_NAME_UPDATED, METRIC_NAME_WRITTEN, METRIC_OUTPUT, METRIC_READ, METRIC_REJECTED, METRIC_UPDATED, METRIC_WRITTEN, pipelineMeta, pipelineRunConfiguration, pluginId, resultFiles, resultRows, rowsets, rowSetSize, STRING_FINISHED, STRING_FINISHED_WITH_ERRORS, STRING_HALTING, STRING_INITIALIZING, STRING_PAUSED, STRING_PREPARING, STRING_RUNNING, STRING_STOPPED, STRING_STOPPED_WITH_ERRORS, STRING_WAITING, TYPE_DISP_1_1, TYPE_DISP_1_N, TYPE_DISP_N_1, TYPE_DISP_N_M, TYPE_DISP_N_N

Fields inherited from interface org.apache.hop.pipeline.engine.IPipelineEngine

PIPELINE_IN_PREVIEW_MODE

Constructor Summary

Constructors

Constructor	Description
LocalPipelineEngine()
LocalPipelineEngine(PipelineMeta pipelineMeta)
LocalPipelineEngine(PipelineMeta pipelineMeta, IVariables variables, ILoggingObject parent)
LocalPipelineEngine(Parent parent, String name, String filename, IHopMetadataProvider metadataProvider)

Method Summary

Modifier and Type	Method	Description
void	addTransformExecutionSamplers()
IPipelineEngineRunConfiguration	createDefaultPipelineEngineRunConfiguration()	Ask the engine to generate a default pipeline engine configuration for this engine
PipelineEngineCapabilities	getEngineCapabilities()	Gets engineCapabilities
String	getStatusDescription()
void	lookupExecutionInformationLocation()	This method looks up the execution information location specified in the run configuration.
void	pipelineCompleted()	This method is called when all the finished listeners are executed and when the pipeline has completed.
void	prepareExecution()	Prepares the pipeline for execution.
void	registerPipelineExecutionInformation()	If needed, register this pipeline at the specified execution information location.
void	setEngineCapabilities(PipelineEngineCapabilities engineCapabilities)
void	startThreads()	Starts the threads prepared by prepareThreads().
void	startTransformExecutionInfoTimer()
void	stopTransformExecutionInfoTimer()
void	waitUntilFinished()	Waits until all RunThreads have finished.

Methods inherited from class org.apache.hop.pipeline.Pipeline

activateParameters, addActiveSubPipeline, addActiveSubWorkflow, addExecutionDataSampler, addExecutionFinishedListener, addExecutionStartedListener, addExecutionStoppedListener, addFinishedListener, addParameterDefinition, addPipelineStoppedListener, addRowProducer, addStartedListener, addTransformPerformanceSnapShot, beginProcessing, calculateBatchIdAndDateRange, cleanup, clearError, clearParameterValues, copyFrom, copyParametersFromDefinitions, doTopologySortOfTransforms, execute, findComponent, findRowSet, findRunThread, firePipelineExecutionFinishedListeners, firePipelineExecutionStartedListeners, firePipelineExecutionStoppedListeners, getActiveSubPipeline, getActiveSubPipelines, getActiveSubWorkflow, getActiveSubWorkflows, getComponentCopies, getComponentLogText, getComponents, getContainerId, getDataSamplers, getEngineMetrics, getEngineMetrics, getErrors, getExecutingServer, getExecutingUser, getExecutionEndDate, getExecutionFinishedListeners, getExecutionStartDate, getExecutionStartedListeners, getExecutionStoppedListeners, getExtensionDataMap, getFeedbackSize, getFilename, getLogChannel, getLogChannelId, getLoggingHierarchy, getLogLevel, getMetadataProvider, getName, getObjectCopy, getObjectName, getObjectType, getParameterDefault, getParameterDescription, getParameterValue, getParent, getParentPipeline, getParentVariables, getParentWorkflow, getPipelineMeta, getPipelineRunConfiguration, getPipelineTransformExecutionStatusLookup, getPipelineTransformIsRunningLookup, getPluginId, getPreviousResult, getRegistrationDate, getResult, getResultFiles, getResultRows, getRowsets, getRowSetSize, getRunThread, getRunThread, getServletRequest, getServletResponse, getStatus, getTransform, getTransformData, getTransformPerformanceSnapShots, getTransforms, getTransforms, getVariable, getVariable, getVariableBoolean, getVariableNames, getVariables, hasHaltedComponents, hasTransformStarted, initializeFrom, isFeedbackShown, isFinished, isFinishedOrStopped, isForcingSeparateLogging, isGatheringMetrics, isInitializing, isPaused, isPreparing, isPreview, isReadyToStart, isRunning, isSafeModeEnabled, isSortingTransformsTopologically, isStopped, listParameters, nrTransforms, pauseExecution, removeAllParameters, resolve, resolve, resolve, resumeExecution, retrieveComponentOutput, safeStop, setActiveSubPipelines, setActiveSubWorkflows, setContainerId, setDataSamplers, setExecutingServer, setExecutingUser, setExecutionEndDate, setExecutionFinishedListeners, setExecutionStartDate, setExecutionStartedListeners, setExecutionStoppedListeners, setFeedbackShown, setFeedbackSize, setFinished, setForcingSeparateLogging, setGatheringMetrics, setInitializing, setInternalEntryCurrentDirectory, setInternalHopVariables, setLogChannel, setLogLevel, setMetadataProvider, setParameterValue, setParent, setParentPipeline, setParentVariables, setParentWorkflow, setPaused, setPipelineMeta, setPipelineRunConfiguration, setPluginId, setPreparing, setPreview, setPreviousResult, setReadyToStart, setResultFiles, setResultRows, setRowSetSize, setRunning, setSafeModeEnabled, setServletReponse, setServletRequest, setSortingTransformsTopologically, setStopped, setTransformPerformanceSnapShots, setTransforms, setVariable, setVariables, setVariables, shareWith, stopAll, stopTransform, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hop.core.logging.ILoggingObject

getContainerId, getFilename, getObjectCopy, getObjectName, getObjectType, getParent, getRegistrationDate, isForcingSeparateLogging, isGatheringMetrics, setForcingSeparateLogging, setGatheringMetrics

Methods inherited from interface org.apache.hop.core.parameters.INamedParameterDefinitions

addParameterDefinition, getParameterDefault, getParameterDescription, listParameters

Methods inherited from interface org.apache.hop.core.parameters.INamedParameters

activateParameters, clearParameterValues, copyParametersFromDefinitions, getParameterValue, removeAllParameters, setParameterValue

Methods inherited from interface org.apache.hop.pipeline.engine.IPipelineEngine

addActiveSubPipeline, addActiveSubWorkflow, addExecutionDataSampler, addExecutionFinishedListener, addExecutionStartedListener, addExecutionStoppedListener, cleanup, execute, findComponent, findRowSet, firePipelineExecutionFinishedListeners, firePipelineExecutionStartedListeners, firePipelineExecutionStoppedListeners, getActiveSubPipeline, getActiveSubWorkflow, getComponentCopies, getComponentLogText, getComponents, getEngineMetrics, getEngineMetrics, getErrors, getExecutionEndDate, getExecutionStartDate, getExtensionDataMap, getFeedbackSize, getLogChannel, getLogChannelId, getLogLevel, getMetadataProvider, getParentPipeline, getParentWorkflow, getPipelineMeta, getPipelineRunConfiguration, getPluginId, getPreviousResult, getResult, hasHaltedComponents, isFeedbackShown, isFinished, isPaused, isPreparing, isPreview, isReadyToStart, isRunning, isSafeModeEnabled, isStopped, pauseExecution, resumeExecution, retrieveComponentOutput, setContainerId, setInternalHopVariables, setLogChannel, setLogLevel, setMetadataProvider, setParent, setParentPipeline, setParentWorkflow, setPipelineMeta, setPipelineRunConfiguration, setPluginId, setPreview, setPreviousResult, stopAll

Methods inherited from interface org.apache.hop.core.variables.IVariables

copyFrom, getParentVariables, getVariable, getVariable, getVariableBoolean, getVariableNames, initializeFrom, resolve, resolve, resolve, setParentVariables, setVariable, setVariables, shareWith

Constructor Detail

LocalPipelineEngine

public LocalPipelineEngine()

LocalPipelineEngine

public LocalPipelineEngine(PipelineMeta pipelineMeta)

LocalPipelineEngine

public LocalPipelineEngine(PipelineMeta pipelineMeta,
                           IVariables variables,
                           ILoggingObject parent)

LocalPipelineEngine

public LocalPipelineEngine(Parent parent,
                           String name,
                           String filename,
                           IHopMetadataProvider metadataProvider)
                    throws HopException

Throws:

HopException

Method Detail

createDefaultPipelineEngineRunConfiguration

public IPipelineEngineRunConfiguration createDefaultPipelineEngineRunConfiguration()

Description copied from interface: IPipelineEngine

Ask the engine to generate a default pipeline engine configuration for this engine

Specified by:

createDefaultPipelineEngineRunConfiguration in interface IPipelineEngine<PipelineMeta>

Overrides:

createDefaultPipelineEngineRunConfiguration in class Pipeline

Returns:

a new pipeline engine run configuration

prepareExecution

public void prepareExecution()
                      throws HopException

Description copied from class: Pipeline

Prepares the pipeline for execution. This includes setting the arguments and parameters as well as preparing and tracking the transforms and hops in the pipeline.

Specified by:

prepareExecution in interface IPipelineEngine<PipelineMeta>

Overrides:

prepareExecution in class Pipeline

Throws:

HopException - in case the pipeline could not be prepared (initialized)

registerPipelineExecutionInformation

public void registerPipelineExecutionInformation()
                                          throws HopException

If needed, register this pipeline at the specified execution information location. The name of the location is specified in the run configuration.

Throws:

HopException - In case something goes wrong

addTransformExecutionSamplers

public void addTransformExecutionSamplers()
                                   throws HopException

Throws:

HopException

startThreads

public void startThreads()
                  throws HopException

Description copied from class: Pipeline

Starts the threads prepared by prepareThreads(). Before you start the threads, you can add RowListeners to them.

Specified by:

startThreads in interface IPipelineEngine<PipelineMeta>

Overrides:

startThreads in class Pipeline

Throws:

HopException - if there is a communication error with a remote output socket.

waitUntilFinished

public void waitUntilFinished()

Description copied from class: Pipeline

Waits until all RunThreads have finished.

Specified by:

waitUntilFinished in interface IPipelineEngine<PipelineMeta>

Overrides:

waitUntilFinished in class Pipeline

startTransformExecutionInfoTimer

public void startTransformExecutionInfoTimer()
                                      throws HopException

Throws:

HopException

lookupExecutionInformationLocation

public void lookupExecutionInformationLocation()
                                        throws HopException

This method looks up the execution information location specified in the run configuration.

Throws:

HopException - In case something fundamental went wrong.

pipelineCompleted

public void pipelineCompleted()
                       throws HopException

Description copied from interface: IPipelineEngine

This method is called when all the finished listeners are executed and when the pipeline has completed.

Specified by:

pipelineCompleted in interface IPipelineEngine<PipelineMeta>

Throws:

HopException

stopTransformExecutionInfoTimer

public void stopTransformExecutionInfoTimer()

getEngineCapabilities

public PipelineEngineCapabilities getEngineCapabilities()

Gets engineCapabilities

Specified by:

getEngineCapabilities in interface IPipelineEngine<PipelineMeta>

Returns:

value of engineCapabilities

setEngineCapabilities

public void setEngineCapabilities(PipelineEngineCapabilities engineCapabilities)

Parameters:

engineCapabilities - The engineCapabilities to set

getStatusDescription

public String getStatusDescription()

Specified by:

getStatusDescription in interface IPipelineEngine<PipelineMeta>

Returns:

Get a status description of the state of the engine (running, stopped, finished, paused, halted, ...)