weka.knowledgeflow.steps

Class Appender

java.lang.Object

weka.knowledgeflow.steps.BaseStep

weka.knowledgeflow.steps.Appender

All Implemented Interfaces:

java.io.Serializable, BaseStepExtender, Step

@KFStep(name="Appender",
        category="Flow",
        toolTipText="Append multiple sets of instances",
        iconPath="weka/gui/knowledgeflow/icons/Appender.png")
public class Appender
extends BaseStep

A bean that appends multiple incoming data connections into a single data set. The incoming connections can be either all instance connections or all batch-oriented connections (i.e. data set, training set and test set). Instance and batch connections can't be mixed. An amalgamated output is created that is a combination of all the incoming attributes. Missing values are used to fill columns that don't exist in a particular incoming data set. If all incoming connections are instance connections, then the outgoing connection must be an instance connection (and vice versa for incoming batch connections).

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.Map<Step,Instances>	m_completed Used to keep track of how many upstream steps have sent us complete data sets (batch) or headers (incremental) so far.
protected Instances	m_completeHeader Used to hold the final header in the case of incremental operation
protected java.util.Map<Step,java.io.File>	m_incrementalFiles Holds a files in play for incremental incoming connections
protected java.util.Map<Step,java.io.ObjectOutputStream>	m_incrementalSavers Holds savers used for incrementally saving incoming instance streams.
protected boolean	m_isReset True if this step has been reset
protected java.util.concurrent.atomic.AtomicInteger	m_streamingCountDown Gets decremented for each incoming instance stream that has finished
protected Data	m_streamingData Re-usable data object for streaming mode
protected java.util.Map<Step,java.io.File>	m_tempBatchFiles Handles on temp files used to store batches of instances in batch mode

Fields inherited from class weka.knowledgeflow.steps.BaseStep

m_stepIsResourceIntensive, m_stepManager, m_stepName

Constructor Summary

Constructors

Constructor and Description
Appender()

Method Summary

Modifier and Type	Method and Description
java.util.List<java.lang.String>	getIncomingConnectionTypes() Get the incoming connection types accepted by this step at this time
java.util.List<java.lang.String>	getOutgoingConnectionTypes() Get a list of outgoing connection types that this step can produce at this time
protected Instances	makeOutputHeader() Create the structure of the output
protected Instances	makeOutputHeader(java.util.Collection<Instances> headers) Create the structure of the output given a collection of input structures
Instances	outputStructureForConnectionType(java.lang.String connectionName) If possible, get the output structure for the named connection type as a header-only set of instances.
protected void	processBatch(Data data) Process batch data
void	processIncoming(Data data) Process an incoming data payload (if the step accepts incoming connections)
protected void	processStreaming(Data data) Process streaming data
void	stepInit() Initialize the step

Methods inherited from class weka.knowledgeflow.steps.BaseStep

environmentSubstitute, getCustomEditorForStep, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

m_completed

protected java.util.Map<Step,Instances> m_completed

Used to keep track of how many upstream steps have sent us complete data sets (batch) or headers (incremental) so far.

m_tempBatchFiles

protected java.util.Map<Step,java.io.File> m_tempBatchFiles

Handles on temp files used to store batches of instances in batch mode

m_completeHeader

protected Instances m_completeHeader

Used to hold the final header in the case of incremental operation

m_streamingCountDown

protected java.util.concurrent.atomic.AtomicInteger m_streamingCountDown

Gets decremented for each incoming instance stream that has finished

m_incrementalSavers

protected transient java.util.Map<Step,java.io.ObjectOutputStream> m_incrementalSavers

Holds savers used for incrementally saving incoming instance streams. After we've seen the structure from each incoming connection we can create the final output structure, pull any saved instances from the temp files and discard these savers as they will no longer be needed.

m_incrementalFiles

protected transient java.util.Map<Step,java.io.File> m_incrementalFiles

Holds a files in play for incremental incoming connections

m_streamingData

protected Data m_streamingData

Re-usable data object for streaming mode

m_isReset

protected boolean m_isReset

True if this step has been reset

Constructor Detail

Appender

public Appender()

Method Detail

stepInit

public void stepInit()
              throws WekaException

Initialize the step

Throws:

WekaException - if a problem occurs

getIncomingConnectionTypes

public java.util.List<java.lang.String> getIncomingConnectionTypes()

Get the incoming connection types accepted by this step at this time

Returns:

a list of incoming connection types

getOutgoingConnectionTypes

public java.util.List<java.lang.String> getOutgoingConnectionTypes()

Get a list of outgoing connection types that this step can produce at this time

Returns:

a list of outgoing connection types

processIncoming

public void processIncoming(Data data)
                     throws WekaException

Process an incoming data payload (if the step accepts incoming connections)

Specified by:

processIncoming in interface BaseStepExtender

Specified by:

processIncoming in interface Step

Overrides:

processIncoming in class BaseStep

Parameters:

data - the data to process

Throws:

WekaException - if a problem occurs

processBatch

protected void processBatch(Data data)
                     throws WekaException

Process batch data

Parameters:

data - the data to process

Throws:

WekaException - if a problem occurs

processStreaming

protected void processStreaming(Data data)
                         throws WekaException

Process streaming data

Parameters:

data - the data to process

Throws:

WekaException - if a problem occurs

makeOutputHeader

protected Instances makeOutputHeader()
                              throws WekaException

Create the structure of the output

Returns:

the structure of the output as a header-only set of instances

Throws:

WekaException - if a problem occurs

makeOutputHeader

protected Instances makeOutputHeader(java.util.Collection<Instances> headers)
                              throws WekaException

Create the structure of the output given a collection of input structures

Parameters:

headers - a collection of incoming instance structures

Returns:

the structure of the output as a header-only set of instances

Throws:

WekaException - if a problem occurs

outputStructureForConnectionType

public Instances outputStructureForConnectionType(java.lang.String connectionName)
                                           throws WekaException

If possible, get the output structure for the named connection type as a header-only set of instances. Can return null if the specified connection type is not representable as Instances or cannot be determined at present.

Specified by:

outputStructureForConnectionType in interface Step

Overrides:

outputStructureForConnectionType in class BaseStep

Parameters:

connectionName - the name of the connection to get the output structure for

Returns:

the output structure or null if it can't be produced

Throws:

WekaException - if a problem occurs