weka.filters.unsupervised.instance

Class RemoveDuplicates

java.lang.Object

weka.filters.Filter

weka.filters.SimpleFilter

weka.filters.SimpleBatchFilter

weka.filters.unsupervised.instance.RemoveDuplicates

All Implemented Interfaces:

java.io.Serializable, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, WeightedAttributesHandler, WeightedInstancesHandler

public class RemoveDuplicates
extends SimpleBatchFilter
implements WeightedAttributesHandler, WeightedInstancesHandler

Removes all duplicate instances from the first batch of data it receives.

Valid options are:

 -D
  Turns on output of debugging information.

See Also:

Serialized Form

Field Summary

Fields inherited from class weka.filters.Filter

m_Debug, m_DoNotCheckCapabilities, m_FirstBatchDone, m_InputRelAtts, m_InputStringAtts, m_NewBatch, m_OutputRelAtts, m_OutputStringAtts

Constructor Summary

Constructors

Constructor and Description
RemoveDuplicates()

Method Summary

Modifier and Type	Method and Description
protected Instances	determineOutputFormat(Instances inputFormat) Determines the output format based on the input format and returns this.
Capabilities	getCapabilities() Returns the Capabilities of this filter.
java.lang.String	getRevision() Returns the revision string.
java.lang.String	globalInfo() Returns a string describing this filter.
protected boolean	hasImmediateOutputFormat() returns true if the output format is immediately available after the input format has been set and not only after all the data has been seen (see batchFinished())
boolean	input(Instance instance) Input an instance for filtering.
static void	main(java.lang.String[] args) Main method for running this filter.
protected Instances	process(Instances instances) Processes the given data (may change the provided dataset) and returns the modified version.

Methods inherited from class weka.filters.SimpleBatchFilter

allowAccessToFullInputFormat, batchFinished

Methods inherited from class weka.filters.SimpleFilter

reset, setInputFormat

Methods inherited from class weka.filters.Filter

batchFilterFile, bufferInput, copyValues, copyValues, debugTipText, doNotCheckCapabilitiesTipText, filterFile, flushInput, getCapabilities, getCopyOfInputFormat, getDebug, getDoNotCheckCapabilities, getInputFormat, getOptions, getOutputFormat, initInputLocators, initOutputLocators, inputFormatPeek, isFirstBatchDone, isNewBatch, isOutputFormatDefined, listOptions, makeCopies, makeCopy, mayRemoveInstanceAfterFirstBatchDone, numPendingOutput, output, outputFormatPeek, outputPeek, postExecution, preExecution, push, push, resetQueue, run, runFilter, setDebug, setDoNotCheckCapabilities, setOptions, setOutputFormat, testInputFormat, toString, useFilter, wekaStaticWrapper

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

RemoveDuplicates

public RemoveDuplicates()

Method Detail

globalInfo

public java.lang.String globalInfo()

Returns a string describing this filter.

Specified by:

globalInfo in class SimpleFilter

Returns:

a description of the filter suitable for displaying in the explorer/experimenter gui

input

public boolean input(Instance instance)
              throws java.lang.Exception

Input an instance for filtering. Filter requires all training instances be read before producing output (calling the method batchFinished() makes the data available). If this instance is part of a new batch, m_NewBatch is set to false.

Overrides:

input in class SimpleBatchFilter

Parameters:

instance - the input instance

Returns:

true if the filtered instance may now be collected with output().

Throws:

java.lang.IllegalStateException - if no input structure has been defined

java.lang.Exception - if something goes wrong

See Also:

SimpleBatchFilter.batchFinished()

getCapabilities

public Capabilities getCapabilities()

Returns the Capabilities of this filter.

Specified by:

getCapabilities in interface CapabilitiesHandler

Overrides:

getCapabilities in class Filter

Returns:

the capabilities of this object

See Also:

Capabilities

determineOutputFormat

protected Instances determineOutputFormat(Instances inputFormat)
                                   throws java.lang.Exception

Determines the output format based on the input format and returns this.

Specified by:

determineOutputFormat in class SimpleFilter

Parameters:

inputFormat - the input format to base the output format on

Returns:

the output format

Throws:

java.lang.Exception - in case the determination goes wrong

See Also:

SimpleFilter.hasImmediateOutputFormat(), Filter.batchFinished()

hasImmediateOutputFormat

protected boolean hasImmediateOutputFormat()

returns true if the output format is immediately available after the input format has been set and not only after all the data has been seen (see batchFinished())

Overrides:

hasImmediateOutputFormat in class SimpleBatchFilter

Returns:

true if the output format is immediately available

See Also:

SimpleBatchFilter.batchFinished(), SimpleFilter.setInputFormat(Instances)

process

protected Instances process(Instances instances)
                     throws java.lang.Exception

Processes the given data (may change the provided dataset) and returns the modified version. This method is called in batchFinished().

Specified by:

process in class SimpleFilter

Parameters:

instances - the data to process

Returns:

the modified data

Throws:

java.lang.Exception - in case the processing goes wrong

See Also:

SimpleBatchFilter.batchFinished()

getRevision

public java.lang.String getRevision()

Returns the revision string.

Specified by:

getRevision in interface RevisionHandler

Overrides:

getRevision in class Filter

Returns:

the revision

main

public static void main(java.lang.String[] args)

Main method for running this filter.

Parameters:

args - arguments for the filter: use -h for help