weka.knowledgeflow

Class BaseExecutionEnvironment

java.lang.Object

weka.knowledgeflow.BaseExecutionEnvironment

All Implemented Interfaces:

ExecutionEnvironment

public class BaseExecutionEnvironment
extends java.lang.Object
implements ExecutionEnvironment

Base class for execution environments

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseExecutionEnvironment.BaseExecutionEnvironmentDefaults Defaults for the base execution environment

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	DESCRIPTION Description of the default execution environment
protected java.util.concurrent.ExecutorService	m_clientExecutorService An executor service that steps can use to do work in parallel (if desired) by using StepTask instances.
protected Environment	m_envVars The environment variables to use
protected java.util.concurrent.ExecutorService	m_executorService This is the main executor service, used by the execution environment to invoke processIncoming() on a step when batch data is passed to it - i.e. a step receiving data will execute in its own thread.
protected FlowExecutor	m_flowExecutor The FlowExecutor that will be running the flow.
protected GraphicalEnvironmentCommandHandler	m_graphicalEnvCommandHandler The handler (if any) for application-level commands in a graphical environment
protected boolean	m_headless Whether the execution environment is headless or not
protected Logger	m_log The log
protected LoggingLevel	m_loggingLevel The level to log at
protected LogManager	m_logHandler Log handler to wrap log in
protected Settings	m_settings The knowledge flow settings to use
protected java.util.concurrent.ExecutorService	m_singleThreadService An executor service with a single worker thread.

Constructor Summary

Constructors

Constructor and Description
BaseExecutionEnvironment()

Method Summary

Modifier and Type	Method and Description
FlowExecutor	getDefaultFlowExecutor() Gets a new instance of the default flow executor suitable for use with this execution environment
Defaults	getDefaultSettings() Get default settings for the base execution environment
java.lang.String	getDescription() Get a description of this execution environment
Environment	getEnvironmentVariables() Get environment variables for this execution environment
FlowExecutor	getFlowExecutor() Get the executor that will actually be responsible for running the flow.
GraphicalEnvironmentCommandHandler	getGraphicalEnvironmentCommandHandler() Get the environment for performing commands at the application-level in a graphical environment.
Logger	getLog() Get the log in use
LoggingLevel	getLoggingLevel() Get the logging level in use
Settings	getSettings() Get knowledge flow settings for this execution environment
boolean	isHeadless() Get whether this execution environment is headless
protected void	launchStartPoint(StepManagerImpl startPoint) Launches a Step (via the startStep() method) in either than standard step executor service or the resource intensive executor service.
protected void	sendDataToStep(StepManagerImpl step, Data... data) Send the supplied data to the specified step.
void	setEnvironmentVariables(Environment env) Set environment variables for this execution environment
void	setFlowExecutor(FlowExecutor executor) Set the executor that will actually be responsible for running the flow.
void	setGraphicalEnvironmentCommandHandler(GraphicalEnvironmentCommandHandler handler) Set the environment for performing commands at the application-level in a graphical environment.
void	setHeadless(boolean headless) Set whether this execution environment is headless
void	setLog(Logger log) Set the log to use
void	setLoggingLevel(LoggingLevel level) Set the logging level to use
void	setSettings(Settings settings) Set knowledge flow settings for this execution environment
protected void	startClientExecutionService(int numThreadsMain, int numThreadsHighLoad) Start the main step executor service and the high cpu load executor service for clients to use to execute StepTask instances in this execution environment.
protected void	stopClientExecutionService() Stop the client executor service
void	stopProcessing() The main point at which to request stop processing of a flow.
<T> java.util.concurrent.Future<ExecutionResult<T>>	submitTask(StepTask<T> stepTask) Submit a task to be run by the execution environment.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DESCRIPTION

public static final java.lang.String DESCRIPTION

Description of the default execution environment

See Also:

Constant Field Values

m_flowExecutor

protected FlowExecutor m_flowExecutor

The FlowExecutor that will be running the flow. This is not guaranteed to be available from this class until the flow is initialized

m_headless

protected boolean m_headless

Whether the execution environment is headless or not

m_graphicalEnvCommandHandler

protected GraphicalEnvironmentCommandHandler m_graphicalEnvCommandHandler

The handler (if any) for application-level commands in a graphical environment

m_envVars

protected transient Environment m_envVars

The environment variables to use

m_settings

protected transient Settings m_settings

The knowledge flow settings to use

m_executorService

protected transient java.util.concurrent.ExecutorService m_executorService

This is the main executor service, used by the execution environment to invoke processIncoming() on a step when batch data is passed to it - i.e. a step receiving data will execute in its own thread. This is not used, however, when streaming data is processed, as it usually the case that processing is lightweight in this case and the gains of running a separate thread for each piece of data probably do not outweigh the overheads involved.

m_clientExecutorService

protected transient java.util.concurrent.ExecutorService m_clientExecutorService

An executor service that steps can use to do work in parallel (if desired) by using StepTask instances. This executor service is intended for high cpu load tasks, and the number of threads used by it should be kept <= number of physical CPU cores

m_singleThreadService

protected transient java.util.concurrent.ExecutorService m_singleThreadService

An executor service with a single worker thread. This is used to execute steps that are marked with the SingleThreadedExecution annotation, which effectively limits one object of the type in question to be executing in the KnowledgeFlow/JVM at any one time.

m_log

protected transient Logger m_log

The log

m_logHandler

protected transient LogManager m_logHandler

Log handler to wrap log in

m_loggingLevel

protected LoggingLevel m_loggingLevel

The level to log at

Constructor Detail

BaseExecutionEnvironment

public BaseExecutionEnvironment()

Method Detail

getDescription

public java.lang.String getDescription()

Get a description of this execution environment

Specified by:

getDescription in interface ExecutionEnvironment

Returns:

a description of this execution environemtn

isHeadless

public boolean isHeadless()

Get whether this execution environment is headless

Specified by:

isHeadless in interface ExecutionEnvironment

Returns:

true if this execution environment is headless

setHeadless

public void setHeadless(boolean headless)

Set whether this execution environment is headless

Specified by:

setHeadless in interface ExecutionEnvironment

Parameters:

headless - true if the execution environment is headless

getGraphicalEnvironmentCommandHandler

public GraphicalEnvironmentCommandHandler getGraphicalEnvironmentCommandHandler()

Get the environment for performing commands at the application-level in a graphical environment.

Specified by:

getGraphicalEnvironmentCommandHandler in interface ExecutionEnvironment

Returns:

the graphical environment command handler, or null if running headless

setGraphicalEnvironmentCommandHandler

public void setGraphicalEnvironmentCommandHandler(GraphicalEnvironmentCommandHandler handler)

Set the environment for performing commands at the application-level in a graphical environment.

Specified by:

setGraphicalEnvironmentCommandHandler in interface ExecutionEnvironment

getEnvironmentVariables

public Environment getEnvironmentVariables()

Get environment variables for this execution environment

Specified by:

getEnvironmentVariables in interface ExecutionEnvironment

Returns:

the environment variables for this execution environment

setEnvironmentVariables

public void setEnvironmentVariables(Environment env)

Set environment variables for this execution environment

Specified by:

setEnvironmentVariables in interface ExecutionEnvironment

Parameters:

env - the environment variables to use

setSettings

public void setSettings(Settings settings)

Description copied from interface: ExecutionEnvironment

Set knowledge flow settings for this execution environment

Specified by:

setSettings in interface ExecutionEnvironment

Parameters:

settings - the settings to use

getSettings

public Settings getSettings()

Description copied from interface: ExecutionEnvironment

Get knowledge flow settings for this execution environment

Specified by:

getSettings in interface ExecutionEnvironment

Returns:

the settings to use

getLog

public Logger getLog()

Get the log in use

Specified by:

getLog in interface ExecutionEnvironment

Returns:

the log in use

setLog

public void setLog(Logger log)

Set the log to use

Specified by:

setLog in interface ExecutionEnvironment

Parameters:

log - the log to use

getLoggingLevel

public LoggingLevel getLoggingLevel()

Get the logging level in use

Specified by:

getLoggingLevel in interface ExecutionEnvironment

Returns:

the logging level in use

setLoggingLevel

public void setLoggingLevel(LoggingLevel level)

Set the logging level to use

Specified by:

setLoggingLevel in interface ExecutionEnvironment

Parameters:

level - the logging level to use

submitTask

public <T> java.util.concurrent.Future<ExecutionResult<T>> submitTask(StepTask<T> stepTask)
                                                               throws WekaException

Submit a task to be run by the execution environment. The default execution environment uses an ExecutorService to run tasks in parallel. Client steps are free to use this service or to just do their processing locally within their own code.

Specified by:

submitTask in interface ExecutionEnvironment

Parameters:

stepTask - the StepTask encapsulating the code to be run

Returns:

the Future holding the status and result when complete

Throws:

WekaException - if processing fails in the case of

stopProcessing

public void stopProcessing()

The main point at which to request stop processing of a flow. This will request the FlowExecutor to stop and then shutdown the executor service

Specified by:

stopProcessing in interface ExecutionEnvironment

getDefaultFlowExecutor

public FlowExecutor getDefaultFlowExecutor()

Gets a new instance of the default flow executor suitable for use with this execution environment

Returns:

a new instance of the default flow executor suitable for use with this execution environment

getFlowExecutor

public FlowExecutor getFlowExecutor()

Get the executor that will actually be responsible for running the flow. This is not guaranteed to be available from this execution environment until the flow is actually running (or at least initialized)

Returns:

the executor that will be running the flow

setFlowExecutor

public void setFlowExecutor(FlowExecutor executor)

Set the executor that will actually be responsible for running the flow. This is not guaranteed to be available from this execution environment until the flow is actually running (or at least initialized)

Parameters:

executor - the executor that will be running the flow

startClientExecutionService

protected void startClientExecutionService(int numThreadsMain,
                                           int numThreadsHighLoad)

Start the main step executor service and the high cpu load executor service for clients to use to execute StepTask instances in this execution environment. Client steps are free to use this service or to just do their processing locally within their own code (in which case the main step executor service is used).

Parameters:

numThreadsMain - the number of threads to use (level of parallelism). <= 0 indicates no limit on parallelism

numThreadsHighLoad - the number of threads to use for the high cpu load executor service (executes StepTask instances)

stopClientExecutionService

protected void stopClientExecutionService()

Stop the client executor service

launchStartPoint

protected void launchStartPoint(StepManagerImpl startPoint)
                         throws WekaException

Launches a Step (via the startStep() method) in either than standard step executor service or the resource intensive executor service. Does not check that the step is actually a start point.

Parameters:

startPoint - the step to launch as a start point

Throws:

WekaException - if a problem occurs

sendDataToStep

protected void sendDataToStep(StepManagerImpl step,
                              Data... data)
                       throws WekaException

Send the supplied data to the specified step. Base implementation just calls processIncoming() on the step directly. Subclasses may opt to do something different (e.g. wrap data and target step to be executed, and then execute remotely).

Parameters:

data - the data to input to the target step

step - the step to receive data

Throws:

WekaException - if a problem occurs

getDefaultSettings

public Defaults getDefaultSettings()

Get default settings for the base execution environment

Specified by:

getDefaultSettings in interface ExecutionEnvironment

Returns:

the default settings