Join (interruptible-weka 0.1.2 API)

java.lang.Object
- weka.knowledgeflow.steps.BaseStep
- - weka.knowledgeflow.steps.Join

All Implemented Interfaces:: java.io.Serializable, BaseStepExtender, Step

@KFStep(name="Join",
        category="Flow",
        toolTipText="Performs an inner join on two incoming datasets/instance streams (IMPORTANT: assumes that both datasets are sorted in ascending order of the key fields). If data is not sorted then usea Sorter step to sort both into ascending order of the key fields. Does not handle the case wherekeys are not unique in one or both inputs.",
        iconPath="weka/gui/knowledgeflow/icons/Join.gif")
public class Join
extends BaseStep

Step that performs an inner join on one or more key fields from two incoming batch or streaming datasets.

See Also:: Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static java.lang.String`	`KEY_SPEC_SEPARATOR` Separator used to separate first and second input key specifications
`protected java.util.concurrent.atomic.AtomicInteger`	`m_count` Used to cycle over the headers in the header pool
`protected java.util.Queue<Sorter.InstanceHolder>`	`m_firstBuffer` Buffer for the first input (capped at 100 for incremental)
`protected boolean`	`m_firstFinished` Whether the first is finished (incremental mode)
`protected StepManager`	`m_firstInput` First source of data
`protected java.lang.String`	`m_firstInputConnectionType` Connection type of the first input
`protected boolean`	`m_firstIsWaiting` True if the first input stream is waiting due to a full buffer (incremental mode only)
`protected Instances`	`m_headerOne` The structure of the first incoming dataset
`protected java.util.List<Instances>`	`m_headerPool` A set of copied outgoing structure instances.
`protected Instances`	`m_headerTwo` The structure of the second incoming dataset
`protected int[]`	`m_keyIndexesOne` Indexes of the key fields for the first input
`protected int[]`	`m_keyIndexesTwo` Indexes of the key fields for the second input
`protected java.lang.String`	`m_keySpec` Holds the internal representation of the key specification
`protected Instances`	`m_mergedHeader` The structure of the outgoing dataset
`protected boolean`	`m_runningIncrementally` True if the step is running incrementally
`protected java.util.Queue<Sorter.InstanceHolder>`	`m_secondBuffer` Buffer for the second input (capped at 100 for incremental)
`protected boolean`	`m_secondFinished` Whether the second is finished (incremental mode)
`protected StepManager`	`m_secondInput` Second source of data
`protected java.lang.String`	`m_secondInputConnectionType` Connection type of the second input
`protected boolean`	`m_secondIsWaiting` True if the second input stream is waiting due to a full buffer (incremental mode only)
`protected Data`	`m_streamingData` Reusable data object for streaming output
`protected java.util.Map<java.lang.String,java.lang.Integer>`	`m_stringAttIndexesOne` Holds indexes of string attributes, keyed by attribute name
`protected java.util.Map<java.lang.String,java.lang.Integer>`	`m_stringAttIndexesTwo` Holds indexes of string attributes, keyed by attribute name
`protected boolean`	`m_stringAttsPresent` True if string attributes are present in the incoming data

Fields inherited from class weka.knowledgeflow.steps.BaseStep
m_stepIsResourceIntensive, m_stepManager, m_stepName

Constructor Summary

Constructors
Constructor and Description

Join()

Constructors
Constructor and Description
`Join()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`addToFirstBuffer(Instance inst)` Add an instance to the first buffer
`protected void`	`addToSecondBuffer(Instance inst)` Add an instance to the second buffer
`protected void`	`clearBuffers()` Clear the buffers
`protected int`	`compare(Instance one, Instance two, Sorter.InstanceHolder oneH, Sorter.InstanceHolder twoH)` Compares two instances according to the keys
`protected void`	`establishFirstAndSecondConnectedInputs()` Look for, and configure with respect to, first and second inputs
`protected void`	`generateMergedHeader()` Generate the header of the output instance structure
`protected Instance`	`generateMergedInstance(Sorter.InstanceHolder one, Sorter.InstanceHolder two)` Generate a merged instance from two input instances that match on the key fields
`java.util.List<java.lang.String>`	`getConnectedInputNames()` Get the names of the connected steps as a list
`java.lang.String`	`getCustomEditorForStep()` Return the fully qualified name of a custom editor component (JComponent) to use for editing the properties of the step.
`Instances`	`getFirstInputStructure()` Get the Instances structure being produced by the first input
`java.util.List<java.lang.String>`	`getIncomingConnectionTypes()` Get a list of incoming connection types that this step can accept.
`java.lang.String`	`getKeySpec()` Get the key specification (in internal format - k11,k12,...
`java.util.List<java.lang.String>`	`getOutgoingConnectionTypes()` Get a list of outgoing connection types that this step can produce.
`Instances`	`getSecondInputStructure()` Get the Instances structure being produced by the second input
`protected void`	`processBatch(Data data)` Process batch data.
`protected Instance`	`processBuffers()` Check both buffers and return a joined instance (if possible at this time) or null
`void`	`processIncoming(Data data)` Process some incoming data
`protected void`	`processStreaming(Data data)` Handle streaming data
`void`	`setKeySpec(java.lang.String ks)` Set the key specification (in internal format - k11,k12,...
`void`	`stepInit()` Initialize the step

Methods inherited from class weka.knowledgeflow.steps.BaseStep
environmentSubstitute, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - KEY_SPEC_SEPARATOR
```
public static final java.lang.String KEY_SPEC_SEPARATOR
```
    Separator used to separate first and second input key specifications
    
    See Also:
    
    Constant Field Values
  - m_firstInput
```
protected StepManager m_firstInput
```
    First source of data
  - m_secondInput
```
protected StepManager m_secondInput
```
    Second source of data
  - m_firstFinished
```
protected transient boolean m_firstFinished
```
    Whether the first is finished (incremental mode)
  - m_secondFinished
```
protected transient boolean m_secondFinished
```
    Whether the second is finished (incremental mode)
  - m_firstInputConnectionType
```
protected java.lang.String m_firstInputConnectionType
```
    Connection type of the first input
  - m_secondInputConnectionType
```
protected java.lang.String m_secondInputConnectionType
```
    Connection type of the second input
  - m_firstBuffer
```
protected transient java.util.Queue<Sorter.InstanceHolder> m_firstBuffer
```
    Buffer for the first input (capped at 100 for incremental)
  - m_secondBuffer
```
protected transient java.util.Queue<Sorter.InstanceHolder> m_secondBuffer
```
    Buffer for the second input (capped at 100 for incremental)
  - m_streamingData
```
protected Data m_streamingData
```
    Reusable data object for streaming output
  - m_headerOne
```
protected transient Instances m_headerOne
```
    The structure of the first incoming dataset
  - m_headerTwo
```
protected transient Instances m_headerTwo
```
    The structure of the second incoming dataset
  - m_mergedHeader
```
protected transient Instances m_mergedHeader
```
    The structure of the outgoing dataset
  - m_headerPool
```
protected transient java.util.List<Instances> m_headerPool
```
    A set of copied outgoing structure instances. Used when there are string attributes present in the incremental case in order to prevent concurrency problems where string values could get clobbered in the header.
  - m_count
```
protected transient java.util.concurrent.atomic.AtomicInteger m_count
```
    Used to cycle over the headers in the header pool
  - m_stringAttsPresent
```
protected boolean m_stringAttsPresent
```
    True if string attributes are present in the incoming data
  - m_runningIncrementally
```
protected boolean m_runningIncrementally
```
    True if the step is running incrementally
  - m_keyIndexesOne
```
protected int[] m_keyIndexesOne
```
    Indexes of the key fields for the first input
  - m_keyIndexesTwo
```
protected int[] m_keyIndexesTwo
```
    Indexes of the key fields for the second input
  - m_keySpec
```
protected java.lang.String m_keySpec
```
    Holds the internal representation of the key specification
  - m_stringAttIndexesOne
```
protected java.util.Map<java.lang.String,java.lang.Integer> m_stringAttIndexesOne
```
    Holds indexes of string attributes, keyed by attribute name
  - m_stringAttIndexesTwo
```
protected java.util.Map<java.lang.String,java.lang.Integer> m_stringAttIndexesTwo
```
    Holds indexes of string attributes, keyed by attribute name
  - m_firstIsWaiting
```
protected boolean m_firstIsWaiting
```
    True if the first input stream is waiting due to a full buffer (incremental mode only)
  - m_secondIsWaiting
```
protected boolean m_secondIsWaiting
```
    True if the second input stream is waiting due to a full buffer (incremental mode only)
- Constructor Detail
  - Join
```
public Join()
```
- Method Detail
  - setKeySpec
```
public void setKeySpec(java.lang.String ks)
```
    Set the key specification (in internal format - k11,k12,...,k1nKEY_SPEC_SEPARATORk21,k22,...,k2n)
    
    Parameters:
    
    ks - the keys specification
  - getKeySpec
```
public java.lang.String getKeySpec()
```
    Get the key specification (in internal format - k11,k12,...,k1nKEY_SPEC_SEPARATORk21,k22,...,k2n)
    
    Returns:
    
    the keys specification
  - getConnectedInputNames
```
public java.util.List<java.lang.String> getConnectedInputNames()
```
    Get the names of the connected steps as a list
    
    Returns:
    
    the names of the connected steps as a list
  - getFirstInputStructure
```
public Instances getFirstInputStructure()
                                 throws WekaException
```
    Get the Instances structure being produced by the first input
    
    Returns:
    
    the Instances structure from the first input
    
    Throws:
    
    WekaException - if a problem occurs
  - getSecondInputStructure
```
public Instances getSecondInputStructure()
                                  throws WekaException
```
    Get the Instances structure being produced by the second input
    
    Returns:
    
    the Instances structure from the second input
    
    Throws:
    
    WekaException - if a problem occurs
  - establishFirstAndSecondConnectedInputs
```
protected void establishFirstAndSecondConnectedInputs()
```
    Look for, and configure with respect to, first and second inputs
  - stepInit
```
public void stepInit()
              throws WekaException
```
    Initialize the step
    
    Throws:
    
    WekaException - if a problem occurs
  - processIncoming
```
public void processIncoming(Data data)
                     throws WekaException
```
    Process some incoming data
    
    Specified by:
    
    processIncoming in interface BaseStepExtender
    
    Specified by:
    
    processIncoming in interface Step
    
    Overrides:
    
    processIncoming in class BaseStep
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException - if a problem occurs
  - processStreaming
```
protected void processStreaming(Data data)
                         throws WekaException
```
    Handle streaming data
    
    Parameters:
    
    data - an instance of streaming data
    
    Throws:
    
    WekaException - if a problem occurs
  - addToFirstBuffer
```
protected void addToFirstBuffer(Instance inst)
```
    Add an instance to the first buffer
    
    Parameters:
    
    inst - the instance to add
  - addToSecondBuffer
```
protected void addToSecondBuffer(Instance inst)
```
    Add an instance to the second buffer
    
    Parameters:
    
    inst - the instance to add
  - clearBuffers
```
protected void clearBuffers()
                     throws WekaException
```
    Clear the buffers
    
    Throws:
    
    WekaException - if a problem occurs
  - processBatch
```
protected void processBatch(Data data)
                     throws WekaException
```
    Process batch data.
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException - if a problem occurs
  - processBuffers
```
protected Instance processBuffers()
```
    Check both buffers and return a joined instance (if possible at this time) or null
    
    Returns:
    
    a joined instance or null
  - compare
```
protected int compare(Instance one,
                      Instance two,
                      Sorter.InstanceHolder oneH,
                      Sorter.InstanceHolder twoH)
```
    Compares two instances according to the keys
    
    Parameters:
    
    one - the first instance
    
    two - the second instance
    
    oneH - the first instance holder (in case string attributes are present and we are running incrementally)
    
    twoH - the second instance holder
    
    Returns:
    
    the comparison according to the keys
  - generateMergedInstance
```
protected Instance generateMergedInstance(Sorter.InstanceHolder one,
                                          Sorter.InstanceHolder two)
```
    Generate a merged instance from two input instances that match on the key fields
    
    Parameters:
    
    one - the first input instance
    
    two - the second input instance
    
    Returns:
    
    the merged instance
  - generateMergedHeader
```
protected void generateMergedHeader()
                             throws WekaException
```
    Generate the header of the output instance structure
    
    Throws:
    
    WekaException
  - getIncomingConnectionTypes
```
public java.util.List<java.lang.String> getIncomingConnectionTypes()
```
    Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
    
    Returns:
    
    a list of incoming connections that this step can accept given its current state
  - getOutgoingConnectionTypes
```
public java.util.List<java.lang.String> getOutgoingConnectionTypes()
```
    Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
    
    Returns:
    
    a list of outgoing connections that this step can produce
  - getCustomEditorForStep
```
public java.lang.String getCustomEditorForStep()
```
    Return the fully qualified name of a custom editor component (JComponent) to use for editing the properties of the step. This method can return null, in which case the system will dynamically generate an editor using the GenericObjectEditor
    
    Specified by:
    
    getCustomEditorForStep in interface Step
    
    Overrides:
    
    getCustomEditorForStep in class BaseStep
    
    Returns:
    
    the fully qualified name of a step editor component

Class Join

Field Summary

Fields inherited from class weka.knowledgeflow.steps.BaseStep

Constructor Summary

Method Summary

Methods inherited from class weka.knowledgeflow.steps.BaseStep

Methods inherited from class java.lang.Object

Field Detail

KEY_SPEC_SEPARATOR

m_firstInput

m_secondInput

m_firstFinished

m_secondFinished

m_firstInputConnectionType

m_secondInputConnectionType

m_firstBuffer

m_secondBuffer

m_streamingData

m_headerOne

m_headerTwo

m_mergedHeader

m_headerPool

m_count

m_stringAttsPresent

m_runningIncrementally

m_keyIndexesOne

m_keyIndexesTwo

m_keySpec

m_stringAttIndexesOne

m_stringAttIndexesTwo

m_firstIsWaiting

m_secondIsWaiting

Constructor Detail

Join

Method Detail

setKeySpec

getKeySpec

getConnectedInputNames

getFirstInputStructure

getSecondInputStructure

establishFirstAndSecondConnectedInputs

stepInit

processIncoming

processStreaming

addToFirstBuffer

addToSecondBuffer

clearBuffers

processBatch

processBuffers

compare

generateMergedInstance

generateMergedHeader

getIncomingConnectionTypes

getOutgoingConnectionTypes

getCustomEditorForStep