weka.filters

Class SimpleStreamFilter

java.lang.Object

weka.filters.Filter

weka.filters.SimpleFilter

weka.filters.SimpleStreamFilter

All Implemented Interfaces:

java.io.Serializable, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, StreamableFilter

Direct Known Subclasses:

ClassAssigner, FixedDictionaryStringToWordVector, MultiFilter, NumericCleaner, RandomSubset, RemoveByName, RenameAttribute, SortLabels

public abstract class SimpleStreamFilter
extends SimpleFilter
implements StreamableFilter

This filter is a superclass for simple stream filters.

General notes:

• After the first call of batchFinished() the field m_FirstBatchDone is set to true.

Example:
The following code snippet uses the filter SomeFilter on a dataset that is loaded from filename.

 import weka.core.*;
 import weka.filters.*;
 import java.io.*;
 ...
 SomeFilter filter = new SomeFilter();
 // set necessary options for the filter
 Instances data = new Instances(
                    new BufferedReader(
                      new FileReader(filename)));
 Instances filteredData = Filter.useFilter(data, filter);
 

Implementation:
Only the following abstract methods need to be implemented:

• globalInfo()
• determineOutputFormat(Instances)
• process(Instance)

And the getCapabilities() method must return what kind of attributes and classes the filter can handle.

If more options are necessary, then the following methods need to be overriden:

• listOptions()
• setOptions(String[])
• getOptions()

To make the filter available from commandline one must add the following main method for correct execution (<Filtername> must be replaced with the actual filter classname):

  public static void main(String[] args) {
    runFilter(new <Filtername>(), args);
  }
 

Example implementation:

 import weka.core.*;
 import weka.core.Capabilities.*;
 import weka.filters.*;
 
 import java.util.Random;
 
 public class SimpleStream extends SimpleStreamFilter {
 
   public String globalInfo() {
     return "A simple stream filter that adds an attribute 'bla' at the end containing a random number.";
   }
 
   public Capabilities getCapabilities() {
     Capabilities result = super.getCapabilities();
     result.enableAllAttributes();
     result.enableAllClasses();
     result.enable(Capability.NO_CLASS); // filter doesn't need class to be set
     return result;
   }
 
   protected Instances determineOutputFormat(Instances inputFormat) {
     Instances result = new Instances(inputFormat, 0);
     result.insertAttributeAt(new Attribute("bla"), result.numAttributes());
     return result;
   }
 
   protected Instance process(Instance inst) {
     double[] values = new double[inst.numAttributes() + 1];
     for (int n = 0; n < inst.numAttributes(); n++)
       values[n] = inst.value(n);
     values[values.length - 1] = new Random().nextInt();
     Instance result = new DenseInstance(1, values);
     return result;
   }
 
   public static void main(String[] args) {
     runFilter(new SimpleStream(), args);
   }
 }
 
 

Options:
Valid filter-specific options are:

-D
Turns on output of debugging information.

See Also:

SimpleBatchFilter, input(Instance), batchFinished(), Filter.m_FirstBatchDone, Serialized Form

Field Summary

Fields inherited from class weka.filters.Filter

m_Debug, m_DoNotCheckCapabilities, m_FirstBatchDone, m_InputRelAtts, m_InputStringAtts, m_NewBatch, m_OutputRelAtts, m_OutputStringAtts

Constructor Summary

Constructors

Constructor and Description
SimpleStreamFilter()

Method Summary

Modifier and Type	Method and Description
boolean	batchFinished() Signify that this batch of input to the filter is finished.
protected abstract Instances	determineOutputFormat(Instances inputFormat) Determines the output format based on the input format and returns this.
protected boolean	hasImmediateOutputFormat() Returns true if the output format is immediately available after the input format has been set and not only after all the data has been seen (see batchFinished()).
boolean	input(Instance instance) Input an instance for filtering.
protected void	preprocess(Instances instances) In case the output format cannot be returned immediately, this method is called before the actual processing of the instances.
protected abstract Instance	process(Instance instance) processes the given instance (may change the provided instance) and returns the modified version.
protected Instances	process(Instances instances) Processes the given data (may change the provided dataset) and returns the modified version.

Methods inherited from class weka.filters.SimpleFilter

globalInfo, reset, setInputFormat

Methods inherited from class weka.filters.Filter

batchFilterFile, bufferInput, copyValues, copyValues, debugTipText, doNotCheckCapabilitiesTipText, filterFile, flushInput, getCapabilities, getCapabilities, getCopyOfInputFormat, getDebug, getDoNotCheckCapabilities, getInputFormat, getOptions, getOutputFormat, getRevision, initInputLocators, initOutputLocators, inputFormatPeek, isFirstBatchDone, isNewBatch, isOutputFormatDefined, listOptions, main, makeCopies, makeCopy, mayRemoveInstanceAfterFirstBatchDone, numPendingOutput, output, outputFormatPeek, outputPeek, postExecution, preExecution, push, push, resetQueue, run, runFilter, setDebug, setDoNotCheckCapabilities, setOptions, setOutputFormat, testInputFormat, toString, useFilter, wekaStaticWrapper

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

SimpleStreamFilter

public SimpleStreamFilter()

Method Detail

hasImmediateOutputFormat

protected boolean hasImmediateOutputFormat()

Returns true if the output format is immediately available after the input format has been set and not only after all the data has been seen (see batchFinished()). This method should normally return true for a stream filter, since the data will be processed in a batch manner instead (or at least for the second batch of files, see m_FirstBatchDone).

Specified by:

hasImmediateOutputFormat in class SimpleFilter

Returns:

true if the output format is immediately available

See Also:

batchFinished(), SimpleFilter.setInputFormat(Instances), Filter.m_FirstBatchDone

determineOutputFormat

protected abstract Instances determineOutputFormat(Instances inputFormat)
                                            throws java.lang.Exception

Determines the output format based on the input format and returns this. In case the output format cannot be returned immediately, i.e., hasImmediateOutputFormat() returns false, then this method will called from batchFinished() after the call of preprocess(Instances), in which, e.g., statistics for the actual processing step can be gathered.

Specified by:

determineOutputFormat in class SimpleFilter

Parameters:

inputFormat - the input format to base the output format on

Returns:

the output format

Throws:

java.lang.Exception - in case the determination goes wrong

See Also:

hasImmediateOutputFormat(), batchFinished(), preprocess(Instances)

process

protected abstract Instance process(Instance instance)
                             throws java.lang.Exception

processes the given instance (may change the provided instance) and returns the modified version.

Parameters:

instance - the instance to process

Returns:

the modified data

Throws:

java.lang.Exception - in case the processing goes wrong

process

protected Instances process(Instances instances)
                     throws java.lang.Exception

Processes the given data (may change the provided dataset) and returns the modified version. This method is called in batchFinished(). This implementation only calls process(Instance) for each instance in the given dataset.

Specified by:

process in class SimpleFilter

Parameters:

instances - the data to process

Returns:

the modified data

Throws:

java.lang.Exception - in case the processing goes wrong

See Also:

batchFinished(), process(Instance)

preprocess

protected void preprocess(Instances instances)

In case the output format cannot be returned immediately, this method is called before the actual processing of the instances. Derived classes can implement specific behavior here.

Parameters:

instances - the instances to work on

See Also:

hasImmediateOutputFormat(), determineOutputFormat(Instances)

input

public boolean input(Instance instance)
              throws java.lang.Exception

Input an instance for filtering. Filter requires all training instances be read before producing output.

Overrides:

input in class Filter

Parameters:

instance - the input instance

Returns:

true if the filtered instance may now be collected with output().

Throws:

java.lang.IllegalStateException - if no input structure has been defined

java.lang.Exception - if something goes wrong

batchFinished

public boolean batchFinished()
                      throws java.lang.Exception

Signify that this batch of input to the filter is finished. If the filter requires all instances prior to filtering, output() may now be called to retrieve the filtered instances. Any subsequent instances filtered should be filtered based on setting obtained from the first batch (unless the setInputFormat has been re-assigned or new options have been set).

Overrides:

batchFinished in class Filter

Returns:

true if there are instances pending output

Throws:

java.lang.IllegalStateException - if no input format has been set.

java.lang.NullPointerException - if no input structure has been defined,

java.lang.Exception - if there was a problem finishing the batch.