Sorter (interruptible-weka 0.1.2 API)

java.lang.Object
- weka.knowledgeflow.steps.BaseStep
- - weka.knowledgeflow.steps.Sorter

All Implemented Interfaces:: java.io.Serializable, BaseStepExtender, Step

@KFStep(name="Sorter",
        category="Tools",
        toolTipText="Sort instances in ascending or descending order according to the values of user-specified attributes. Instances can be sorted according to multiple attributes (defined in order). Handles datasets larger than can be fit into main memory via instance connections and specifying the in-memory buffer size. Implements a merge-sort by writing the sorted in-memory buffer to a file when full and then interleaving instances from the disk-based file(s) when the incoming stream has finished.",
        iconPath="weka/gui/knowledgeflow/icons/Sorter.gif")
public class Sorter
extends BaseStep

Step for sorting instances according to one or more attributes.

See Also:: Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type	Class and Description
`protected static class`	`Sorter.InstanceHolder` Inner class that holds instances and the index of the temp file that holds them (if operating in incremental mode)
`protected static class`	`Sorter.SortComparator` Comparator that applies the sort rules to `InstanceHolder`s
`static class`	`Sorter.SortRule` Implements a sorting rule based on a single attribute

Field Summary

Fields
Modifier and Type	Field and Description
`protected java.util.List<java.io.File>`	`m_bufferFiles` List of sorted temp files for incremental operation
`protected java.lang.String`	`m_bufferSize` Size of the in-memory buffer
`protected int`	`m_bufferSizeI` Size of the in-memory buffer after resolving any environment vars
`protected Instances`	`m_connectedFormat` format of instances for current incoming connection (if any)
`protected java.util.List<Sorter.InstanceHolder>`	`m_incrementalBuffer` In memory buffer for incremental operation
`protected boolean`	`m_isReset` True if we've been reset
`protected Sorter.SortComparator`	`m_sortComparator` Comparator that applies the sort rules
`protected java.lang.String`	`m_sortDetails` Holds the internal textual description of the sort definitions
`protected boolean`	`m_streaming` True if processing streaming data
`protected Data`	`m_streamingData` To (re)use when streaming
`protected java.util.Map<java.lang.String,java.lang.Integer>`	`m_stringAttIndexes` Holds indexes of string attributes, keyed by attribute name
`protected java.io.File`	`m_tempDirectory` The directory to hold the temp files - if not set the system tmp directory is used

Fields inherited from class weka.knowledgeflow.steps.BaseStep
m_stepIsResourceIntensive, m_stepManager, m_stepName

Constructor Summary

Constructors
Constructor and Description

Sorter()

Constructors
Constructor and Description
`Sorter()`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`protected void`	`emitBufferedInstances()` Output any buffered instances
`java.lang.String`	`getBufferSize()` Get the size of the in-memory buffer
`java.lang.String`	`getCustomEditorForStep()` Return the fully qualified name of a custom editor component (JComponent) to use for editing the properties of the step.
`java.util.List<java.lang.String>`	`getIncomingConnectionTypes()` Get a list of incoming connection types that this step can accept.
`java.util.List<java.lang.String>`	`getOutgoingConnectionTypes()` Get a list of outgoing connection types that this step can produce.
`java.lang.String`	`getSortDetails()` Get the sort rules to use
`java.io.File`	`getTempDirectory()` Get the directory to use for temporary files during incremental operation
`protected void`	`init(Instances structure)` Initialize given the supplied instances structure
`protected void`	`processBatch(Data data)` Process batch data
`void`	`processIncoming(Data data)` Process an incoming data payload (if the step accepts incoming connections)
`protected void`	`processIncremental(Data data)` Process incremental data
`void`	`setBufferSize(java.lang.String buffSize)` Set the size of the in-memory buffer
`void`	`setSortDetails(java.lang.String sortDetails)` Set the sort rules to use
`void`	`setTempDirectory(java.io.File tempDir)` Set the directory to use for temporary files during incremental operation
`void`	`stepInit()` Initialize the step.

Methods inherited from class weka.knowledgeflow.steps.BaseStep
environmentSubstitute, getDefaultSettings, getInteractiveViewers, getInteractiveViewersImpls, getName, getStepManager, globalInfo, isResourceIntensive, isStopRequested, outputStructureForConnectionType, setName, setStepIsResourceIntensive, setStepManager, setStepMustRunSingleThreaded, start, stepMustRunSingleThreaded, stop

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - m_sortComparator
```
protected transient Sorter.SortComparator m_sortComparator
```
    Comparator that applies the sort rules
  - m_incrementalBuffer
```
protected transient java.util.List<Sorter.InstanceHolder> m_incrementalBuffer
```
    In memory buffer for incremental operation
  - m_bufferFiles
```
protected transient java.util.List<java.io.File> m_bufferFiles
```
    List of sorted temp files for incremental operation
  - m_bufferSize
```
protected java.lang.String m_bufferSize
```
    Size of the in-memory buffer
  - m_bufferSizeI
```
protected int m_bufferSizeI
```
    Size of the in-memory buffer after resolving any environment vars
  - m_stringAttIndexes
```
protected java.util.Map<java.lang.String,java.lang.Integer> m_stringAttIndexes
```
    Holds indexes of string attributes, keyed by attribute name
  - m_sortDetails
```
protected java.lang.String m_sortDetails
```
    Holds the internal textual description of the sort definitions
  - m_tempDirectory
```
protected java.io.File m_tempDirectory
```
    The directory to hold the temp files - if not set the system tmp directory is used
  - m_connectedFormat
```
protected Instances m_connectedFormat
```
    format of instances for current incoming connection (if any)
  - m_isReset
```
protected boolean m_isReset
```
    True if we've been reset
  - m_streaming
```
protected boolean m_streaming
```
    True if processing streaming data
  - m_streamingData
```
protected Data m_streamingData
```
    To (re)use when streaming
- Constructor Detail
  - Sorter
```
public Sorter()
```
- Method Detail
  - getBufferSize
```
public java.lang.String getBufferSize()
```
    Get the size of the in-memory buffer
    
    Returns:
    
    the size of the in-memory buffer
  - setBufferSize
```
@OptionMetadata(displayName="Size of in-mem streaming buffer",
                description="Number of instances to sort in memory before writing to a temp file (instance connections only)",
                displayOrder=1)
public void setBufferSize(java.lang.String buffSize)
```
    Set the size of the in-memory buffer
    
    Parameters:
    
    buffSize - the size of the in-memory buffer
  - setTempDirectory
```
@FilePropertyMetadata(fileChooserDialogType=0,
                      directoriesOnly=true)
 @OptionMetadata(displayName="Directory for temp files",
                description="Where to store temporary files when spilling to disk",
                displayOrder=2)
public void setTempDirectory(java.io.File tempDir)
```
    Set the directory to use for temporary files during incremental operation
    
    Parameters:
    
    tempDir - the temp dir to use
  - getTempDirectory
```
public java.io.File getTempDirectory()
```
    Get the directory to use for temporary files during incremental operation
    
    Returns:
    
    the temp dir to use
  - setSortDetails
```
@ProgrammaticProperty
public void setSortDetails(java.lang.String sortDetails)
```
    Set the sort rules to use
    
    Parameters:
    
    sortDetails - the sort rules in internal string representation
  - getSortDetails
```
public java.lang.String getSortDetails()
```
    Get the sort rules to use
    
    Returns:
    
    the sort rules in internal string representation
  - stepInit
```
public void stepInit()
              throws WekaException
```
    Initialize the step.
    
    Throws:
    
    WekaException - if a problem occurs during initialization
  - getIncomingConnectionTypes
```
public java.util.List<java.lang.String> getIncomingConnectionTypes()
```
    Get a list of incoming connection types that this step can accept. Ideally (and if appropriate), this should take into account the state of the step and any existing incoming connections. E.g. a step might be able to accept one (and only one) incoming batch data connection.
    
    Returns:
    
    a list of incoming connections that this step can accept given its current state
  - getOutgoingConnectionTypes
```
public java.util.List<java.lang.String> getOutgoingConnectionTypes()
```
    Get a list of outgoing connection types that this step can produce. Ideally (and if appropriate), this should take into account the state of the step and the incoming connections. E.g. depending on what incoming connection is present, a step might be able to produce a trainingSet output, a testSet output or neither, but not both.
    
    Returns:
    
    a list of outgoing connections that this step can produce
  - init
```
protected void init(Instances structure)
```
    Initialize given the supplied instances structure
    
    Parameters:
    
    structure - the structure to initialize with
  - processIncoming
```
public void processIncoming(Data data)
                     throws WekaException
```
    Process an incoming data payload (if the step accepts incoming connections)
    
    Specified by:
    
    processIncoming in interface BaseStepExtender
    
    Specified by:
    
    processIncoming in interface Step
    
    Overrides:
    
    processIncoming in class BaseStep
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException - if a problem occurs
  - processBatch
```
protected void processBatch(Data data)
                     throws WekaException
```
    Process batch data
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException - if a problem occurs
  - processIncremental
```
protected void processIncremental(Data data)
                           throws WekaException
```
    Process incremental data
    
    Parameters:
    
    data - the data to process
    
    Throws:
    
    WekaException - if a problem occurs
  - emitBufferedInstances
```
protected void emitBufferedInstances()
                              throws WekaException
```
    Output any buffered instances
    
    Throws:
    
    WekaException - if a problem occurs
  - getCustomEditorForStep
```
public java.lang.String getCustomEditorForStep()
```
    Return the fully qualified name of a custom editor component (JComponent) to use for editing the properties of the step. This method can return null, in which case the system will dynamically generate an editor using the GenericObjectEditor
    
    Specified by:
    
    getCustomEditorForStep in interface Step
    
    Overrides:
    
    getCustomEditorForStep in class BaseStep
    
    Returns:
    
    the fully qualified name of a step editor component

Class Sorter

Nested Class Summary

Field Summary

Fields inherited from class weka.knowledgeflow.steps.BaseStep

Constructor Summary

Method Summary

Methods inherited from class weka.knowledgeflow.steps.BaseStep

Methods inherited from class java.lang.Object

Field Detail

m_sortComparator

m_incrementalBuffer

m_bufferFiles

m_bufferSize

m_bufferSizeI

m_stringAttIndexes

m_sortDetails

m_tempDirectory

m_connectedFormat

m_isReset

m_streaming

m_streamingData

Constructor Detail

Sorter

Method Detail

getBufferSize

setBufferSize

setTempDirectory

getTempDirectory

setSortDetails

getSortDetails

stepInit

getIncomingConnectionTypes

getOutgoingConnectionTypes

init

processIncoming

processBatch

processIncremental

emitBufferedInstances

getCustomEditorForStep