MergeNominalValues (interruptible-weka 0.1.2 API)

java.lang.Object
- weka.filters.Filter
- - weka.filters.SimpleFilter
  - - weka.filters.SimpleBatchFilter
    - - weka.filters.supervised.attribute.MergeNominalValues

All Implemented Interfaces:

java.io.Serializable, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, TechnicalInformationHandler, WeightedAttributesHandler, WeightedInstancesHandler, SupervisedFilter
```
public class MergeNominalValues
extends SimpleBatchFilter
implements SupervisedFilter, WeightedInstancesHandler, WeightedAttributesHandler, TechnicalInformationHandler
```
Merges values of all nominal attributes among the specified attributes, excluding the class attribute, using the CHAID method, but without considering re-splitting of merged subsets. It implements Steps 1 and 2 described by Kass (1980), see

Gordon V. Kass (1980). An Exploratory Technique for Investigating Large Quantities of Categorical Data. Applied Statistics. 29(2):119-127.

Once attribute values have been merged, a chi-squared test using the Bonferroni correction is applied to check if the resulting attribute is a valid predictor, based on the Bonferroni multiplier in Equation 3.2 in Kass (1980). If an attribute does not pass this test, all remaining values (if any) are merged. Nevertheless, useless predictors can slip through without being fully merged, e.g. identifier attributes.

The code applies the Yates correction when the chi-squared statistic is computed.

Note that the algorithm is quadratic in the number of attribute values for an attribute.
Valid options are:
```
 -D
  Turns on output of debugging information.
 
```
```
 -L <double>
  The significance level (default: 0.05).
 
```
```
 -R <range>
  Sets list of attributes to act on (or its inverse). 'first and 'last' are accepted as well.'
  E.g.: first-5,7,9,20-last
  (default: first-last)
 
```
```
 -V
  Invert matching sense (i.e. act on all attributes not specified in list)
 
```
```
 -O
  Use short identifiers for merged subsets.
 
```
See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean[]`	`m_AttToBeModified` Indicators for which attributes need to be changed.
`protected int[][]`	`m_Indicators` The indicators used to map the old values.
`protected Range`	`m_SelectCols` Stores which atributes to operate on (or nto)
`protected int[]`	`m_SelectedAttributes` Stores the indexes of the selected attributes in order.
`protected double`	`m_SigLevel` Set the significance level
`protected boolean`	`m_UseShortIdentifiers` Use short values

Fields inherited from class weka.filters.Filter
m_Debug, m_DoNotCheckCapabilities, m_FirstBatchDone, m_InputRelAtts, m_InputStringAtts, m_NewBatch, m_OutputRelAtts, m_OutputStringAtts

Constructor Summary

Constructors
Constructor and Description

MergeNominalValues()

Constructors
Constructor and Description
`MergeNominalValues()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`boolean`	`allowAccessToFullInputFormat()` We need access to the full input data in determineOutputFormat.
`java.lang.String`	`attributeIndicesTipText()` Returns the tip text for this property
`protected double`	`BFfactor(int c, int r)` Compute factor for Bonferroni correction.
`protected Instances`	`determineOutputFormat(Instances inputFormat)` Determines the output format based on the input format and returns this.
`java.lang.String`	`getAttributeIndices()` Get the current range selection.
`Capabilities`	`getCapabilities()` Returns the Capabilities of this filter.
`boolean`	`getInvertSelection()` Get whether the supplied attributes are to be acted on or all other attributes.
`java.lang.String[]`	`getOptions()` Gets the current settings of the filter.
`java.lang.String`	`getRevision()` Returns the revision string.
`double`	`getSignificanceLevel()` Gets the significance level.
`TechnicalInformation`	`getTechnicalInformation()` Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
`boolean`	`getUseShortIdentifiers()` Get whether short identifiers are to be output.
`java.lang.String`	`globalInfo()` Returns a string describing this filter.
`java.lang.String`	`invertSelectionTipText()` Returns the tip text for this property
`java.util.Enumeration<Option>`	`listOptions()` Returns an enumeration describing the available options.
`static void`	`main(java.lang.String[] args)` runs the filter with the given arguments
`protected int[]`	`mergeValues(double[][] counts)` Merges values and returns list of subset indicators for the values.
`protected Instances`	`process(Instances instances)` Processes the given data.
`void`	`setAttributeIndices(java.lang.String rangeList)` Set which attributes are to be acted on (or not, if invert is true)
`void`	`setAttributeIndicesArray(int[] attributes)` Set which attributes are to be acted on (or not, if invert is true)
`void`	`setInvertSelection(boolean invert)` Set whether selected attributes should be acted on or all other attributes.
`void`	`setOptions(java.lang.String[] options)` Parses a given list of options.
`void`	`setSignificanceLevel(double sF)` Sets the significance level.
`void`	`setUseShortIdentifiers(boolean b)` Set whether to output short identifiers for merged values.
`java.lang.String`	`significanceLevelTipText()` Returns the tip text for this property
`java.lang.String`	`useShortIdentifiersTipText()` Returns the tip text for this property

Methods inherited from class weka.filters.SimpleBatchFilter
batchFinished, hasImmediateOutputFormat, input

Methods inherited from class weka.filters.SimpleFilter
reset, setInputFormat

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - m_SigLevel
```
protected double m_SigLevel
```
    Set the significance level
  - m_SelectCols
```
protected Range m_SelectCols
```
    Stores which atributes to operate on (or nto)
  - m_SelectedAttributes
```
protected int[] m_SelectedAttributes
```
    Stores the indexes of the selected attributes in order.
  - m_AttToBeModified
```
protected boolean[] m_AttToBeModified
```
    Indicators for which attributes need to be changed.
  - m_Indicators
```
protected int[][] m_Indicators
```
    The indicators used to map the old values.
  - m_UseShortIdentifiers
```
protected boolean m_UseShortIdentifiers
```
    Use short values
- Constructor Detail
  - MergeNominalValues
```
public MergeNominalValues()
```
- Method Detail
  - globalInfo
```
public java.lang.String globalInfo()
```
    Returns a string describing this filter.
    
    Specified by:
    
    globalInfo in class SimpleFilter
    
    Returns:
    
    a description of the filter suitable for displaying in the explorer/experimenter gui
  - getTechnicalInformation
```
public TechnicalInformation getTechnicalInformation()
```
    Returns an instance of a TechnicalInformation object, containing detailed information about the technical background of this class, e.g., paper reference or book this class is based on.
    
    Specified by:
    
    getTechnicalInformation in interface TechnicalInformationHandler
    
    Returns:
    
    the technical information about this class
  - listOptions
```
public java.util.Enumeration<Option> listOptions()
```
    Returns an enumeration describing the available options.
    
    Specified by:
    
    listOptions in interface OptionHandler
    
    Overrides:
    
    listOptions in class Filter
    
    Returns:
    
    an enumeration of all the available options.
  - getOptions
```
public java.lang.String[] getOptions()
```
    Gets the current settings of the filter.
    
    Specified by:
    
    getOptions in interface OptionHandler
    
    Overrides:
    
    getOptions in class Filter
    
    Returns:
    
    an array of strings suitable for passing to setOptions
  - setOptions
```
public void setOptions(java.lang.String[] options)
                throws java.lang.Exception
```
    Parses a given list of options.
    Valid options are:
```
 -D
  Turns on output of debugging information.
 
```
```
 -L <double>
  The significance level (default: 0.05).
 
```
```
 -R <range>
  Sets list of attributes to act on (or its inverse). 'first and 'last' are accepted as well.'
  E.g.: first-5,7,9,20-last
  (default: first-last)
 
```
```
 -V
  Invert matching sense (i.e. act on all attributes not specified in list)
 
```
```
 -O
  Use short identifiers for merged subsets.
 
```
    Specified by:
    
    setOptions in interface OptionHandler
    
    Overrides:
    
    setOptions in class Filter
    
    Parameters:
    
    options - the list of options as an array of strings
    
    Throws:
    
    java.lang.Exception - if an option is not supported
  - significanceLevelTipText
```
public java.lang.String significanceLevelTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getSignificanceLevel
```
public double getSignificanceLevel()
```
    Gets the significance level.
    
    Returns:
    
    int the significance level.
  - setSignificanceLevel
```
public void setSignificanceLevel(double sF)
```
    Sets the significance level.
    
    Parameters:
    
    sF - the significance level as a double.
  - attributeIndicesTipText
```
public java.lang.String attributeIndicesTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getAttributeIndices
```
public java.lang.String getAttributeIndices()
```
    Get the current range selection.
    
    Returns:
    
    a string containing a comma separated list of ranges
  - setAttributeIndices
```
public void setAttributeIndices(java.lang.String rangeList)
```
    Set which attributes are to be acted on (or not, if invert is true)
    
    Parameters:
    
    rangeList - a string representing the list of attributes. Since the string will typically come from a user, attributes are indexed from 1.
    eg: first-3,5,6-last
  - setAttributeIndicesArray
```
public void setAttributeIndicesArray(int[] attributes)
```
    Set which attributes are to be acted on (or not, if invert is true)
    
    Parameters:
    
    attributes - an array containing indexes of attributes to select. Since the array will typically come from a program, attributes are indexed from 0.
  - invertSelectionTipText
```
public java.lang.String invertSelectionTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getInvertSelection
```
public boolean getInvertSelection()
```
    Get whether the supplied attributes are to be acted on or all other attributes.
    
    Returns:
    
    true if the supplied attributes will be kept
  - setInvertSelection
```
public void setInvertSelection(boolean invert)
```
    Set whether selected attributes should be acted on or all other attributes.
    
    Parameters:
    
    invert - the new invert setting
  - useShortIdentifiersTipText
```
public java.lang.String useShortIdentifiersTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getUseShortIdentifiers
```
public boolean getUseShortIdentifiers()
```
    Get whether short identifiers are to be output.
    
    Returns:
    
    true if short IDs are output
  - setUseShortIdentifiers
```
public void setUseShortIdentifiers(boolean b)
```
    Set whether to output short identifiers for merged values.
    
    Parameters:
    
    b - if true, short IDs are output
  - allowAccessToFullInputFormat
```
public boolean allowAccessToFullInputFormat()
```
    We need access to the full input data in determineOutputFormat.
    
    Overrides:
    
    allowAccessToFullInputFormat in class SimpleBatchFilter
    
    Returns:
    
    whether determineOutputFormat has access to the full input dataset
  - determineOutputFormat
```
protected Instances determineOutputFormat(Instances inputFormat)
```
    Determines the output format based on the input format and returns this.
    
    Specified by:
    
    determineOutputFormat in class SimpleFilter
    
    Parameters:
    
    inputFormat - the input format to base the output format on
    
    Returns:
    
    the output format
    
    See Also:
    
    SimpleFilter.hasImmediateOutputFormat(), Filter.batchFinished()
  - BFfactor
```
protected double BFfactor(int c,
                          int r)
```
    Compute factor for Bonferroni correction. This is based on Equation 3.2 in Kass (1980).
  - mergeValues
```
protected int[] mergeValues(double[][] counts)
```
    Merges values and returns list of subset indicators for the values.
  - getCapabilities
```
public Capabilities getCapabilities()
```
    Returns the Capabilities of this filter.
    
    Specified by:
    
    getCapabilities in interface CapabilitiesHandler
    
    Overrides:
    
    getCapabilities in class Filter
    
    Returns:
    
    the capabilities of this object
    
    See Also:
    
    Capabilities
  - process
```
protected Instances process(Instances instances)
                     throws java.lang.Exception
```
    Processes the given data.
    
    Specified by:
    
    process in class SimpleFilter
    
    Parameters:
    
    instances - the data to process
    
    Returns:
    
    the modified data
    
    Throws:
    
    java.lang.Exception - in case the processing goes wrong
    
    See Also:
    
    Filter.batchFinished()
  - getRevision
```
public java.lang.String getRevision()
```
    Returns the revision string.
    
    Specified by:
    
    getRevision in interface RevisionHandler
    
    Overrides:
    
    getRevision in class Filter
    
    Returns:
    
    the revision
  - main
```
public static void main(java.lang.String[] args)
```
    runs the filter with the given arguments
    
    Parameters:
    
    args - the commandline arguments

Class MergeNominalValues

Field Summary

Fields inherited from class weka.filters.Filter

Constructor Summary

Method Summary

Methods inherited from class weka.filters.SimpleBatchFilter

Methods inherited from class weka.filters.SimpleFilter

Methods inherited from class weka.filters.Filter

Methods inherited from class java.lang.Object

Field Detail

m_SigLevel

m_SelectCols

m_SelectedAttributes

m_AttToBeModified

m_Indicators

m_UseShortIdentifiers

Constructor Detail

MergeNominalValues

Method Detail

globalInfo

getTechnicalInformation

listOptions

getOptions

setOptions

significanceLevelTipText

getSignificanceLevel

setSignificanceLevel

attributeIndicesTipText

getAttributeIndices

setAttributeIndices

setAttributeIndicesArray

invertSelectionTipText

getInvertSelection

setInvertSelection

useShortIdentifiersTipText

getUseShortIdentifiers

setUseShortIdentifiers

allowAccessToFullInputFormat

determineOutputFormat

BFfactor

mergeValues

getCapabilities

process

getRevision

main