SGD (interruptible-weka 0.1.2 API)

java.lang.Object
- weka.classifiers.AbstractClassifier
- - weka.classifiers.RandomizableClassifier
  - - weka.classifiers.functions.SGD

All Implemented Interfaces:: java.io.Serializable, java.lang.Cloneable, Classifier, UpdateableClassifier, Aggregateable<SGD>, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, Randomizable, RevisionHandler

public class SGD
extends RandomizableClassifier
implements UpdateableClassifier, OptionHandler, Aggregateable<SGD>

Implements stochastic gradient descent for learning various linear models (binary class SVM, binary class logistic regression, squared loss, Huber loss and epsilon-insensitive loss linear regression). Globally replaces all missing values and transforms nominal attributes into binary ones. It also normalizes all attributes, so the coefficients in the output are based on the normalized data.
For numeric class attributes, the squared, Huber or epsilon-insensitve loss function must be used. Epsilon-insensitive and Huber loss may require a much higher learning rate.

Valid options are:

  -F
  Set the loss function to minimize.
  0 = hinge loss (SVM), 1 = log loss (logistic regression),
  2 = squared loss (regression), 3 = epsilon insensitive loss (regression),
  4 = Huber loss (regression).
  (default = 0)

  -L
  The learning rate. If normalization is
  turned off (as it is automatically for streaming data), then the
  default learning rate will need to be reduced (try 0.0001).
  (default = 0.01).

  -R <double>
  The lambda regularization constant (default = 0.0001)

  -E <integer>
  The number of epochs to perform (batch learning only, default = 500)

  -C <double>
  The epsilon threshold (epsilon-insenstive and Huber loss only, default = 1e-3)

  -N
  Don't normalize the data

  -M
  Don't replace missing values

  -S <num>
  Random number seed.
  (default 1)

  -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

  -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

See Also:: Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`EPSILON_INSENSITIVE` The epsilon insensitive loss function
`static int`	`HINGE` the hinge loss function.
`static int`	`HUBER` The Huber loss function
`static int`	`LOGLOSS` the log loss function.
`protected Instances`	`m_data` Holds the header of the training data
`protected boolean`	`m_dontNormalize` Turn off normalization of the input data.
`protected boolean`	`m_dontReplaceMissing` Turn off global replacement of missing values.
`protected int`	`m_epochs` The number of epochs to perform (batch learning).
`protected double`	`m_epsilon` The epsilon parameter for epsilon insensitive and Huber loss
`protected double`	`m_lambda` The regularization parameter
`protected double`	`m_learningRate` The learning rate
`protected int`	`m_loss` The current loss function to minimize
`protected Filter`	`m_nominalToBinary` Convert nominal attributes to numerically coded binary ones.
`protected Normalize`	`m_normalize` Normalize the training data
`protected double`	`m_numInstances` The number of training instances
`protected int`	`m_numModels`
`protected ReplaceMissingValues`	`m_replaceMissing` Replace missing values
`protected double`	`m_t` Holds the current iteration number
`protected double[]`	`m_weights` Stores the weights (+ bias in the last element)
`static int`	`SQUAREDLOSS` the squared loss function.
`static Tag[]`	`TAGS_SELECTION` Loss functions to choose from

Fields inherited from class weka.classifiers.RandomizableClassifier
m_Seed

Fields inherited from class weka.classifiers.AbstractClassifier
BATCH_SIZE_DEFAULT, m_BatchSize, m_Debug, m_DoNotCheckCapabilities, m_numDecimalPlaces, NUM_DECIMAL_PLACES_DEFAULT

Constructor Summary

Constructors
Constructor and Description

SGD()

Constructors
Constructor and Description
`SGD()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`SGD`	`aggregate(SGD toAggregate)` Aggregate an object with this one
`void`	`buildClassifier(Instances data)` Method for building the classifier.
`double[]`	`distributionForInstance(Instance inst)` Computes the distribution for a given instance
`protected double`	`dloss(double z)`
`java.lang.String`	`dontNormalizeTipText()` Returns the tip text for this property
`java.lang.String`	`dontReplaceMissingTipText()` Returns the tip text for this property
`protected static double`	`dotProd(Instance inst1, double[] weights, int classIndex)`
`java.lang.String`	`epochsTipText()` Returns the tip text for this property
`java.lang.String`	`epsilonTipText()` Returns the tip text for this property
`void`	`finalizeAggregation()` Call to complete the aggregation process.
`Capabilities`	`getCapabilities()` Returns default capabilities of the classifier.
`boolean`	`getDontNormalize()` Get whether normalization has been turned off.
`boolean`	`getDontReplaceMissing()` Get whether global replacement of missing values has been disabled.
`int`	`getEpochs()` Get current number of epochs
`double`	`getEpsilon()` Get the epsilon threshold on the error for epsilon insensitive and Huber loss functions
`double`	`getLambda()` Get the current value of lambda
`double`	`getLearningRate()` Get the learning rate.
`SelectedTag`	`getLossFunction()` Get the current loss function.
`java.lang.String[]`	`getOptions()` Gets the current settings of the classifier.
`java.lang.String`	`getRevision()` Returns the revision string.
`double[]`	`getWeights()`
`java.lang.String`	`globalInfo()` Returns a string describing classifier
`java.lang.String`	`lambdaTipText()` Returns the tip text for this property
`java.lang.String`	`learningRateTipText()` Returns the tip text for this property
`java.util.Enumeration<Option>`	`listOptions()` Returns an enumeration describing the available options.
`java.lang.String`	`lossFunctionTipText()` Returns the tip text for this property
`static void`	`main(java.lang.String[] args)` Main method for testing this class.
`void`	`reset()` Reset the classifier.
`void`	`setDontNormalize(boolean m)` Turn normalization off/on.
`void`	`setDontReplaceMissing(boolean m)` Turn global replacement of missing values off/on.
`void`	`setEpochs(int e)` Set the number of epochs to use
`void`	`setEpsilon(double e)` Set the epsilon threshold on the error for epsilon insensitive and Huber loss functions
`void`	`setLambda(double lambda)` Set the value of lambda to use
`void`	`setLearningRate(double lr)` Set the learning rate.
`void`	`setLossFunction(SelectedTag function)` Set the loss function to use.
`void`	`setOptions(java.lang.String[] options)` Parses a given list of options.
`java.lang.String`	`toString()` Prints out the classifier.
`void`	`updateClassifier(Instance instance)` Updates the classifier with the given instance.
`protected void`	`updateClassifier(Instance instance, boolean filter)` Updates the classifier with the given instance.

Methods inherited from class weka.classifiers.RandomizableClassifier
getSeed, seedTipText, setSeed

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - m_replaceMissing
```
protected ReplaceMissingValues m_replaceMissing
```
    Replace missing values
  - m_nominalToBinary
```
protected Filter m_nominalToBinary
```
    Convert nominal attributes to numerically coded binary ones. Uses supervised NominalToBinary in the batch learning case
  - m_normalize
```
protected Normalize m_normalize
```
    Normalize the training data
  - m_lambda
```
protected double m_lambda
```
    The regularization parameter
  - m_learningRate
```
protected double m_learningRate
```
    The learning rate
  - m_weights
```
protected double[] m_weights
```
    Stores the weights (+ bias in the last element)
  - m_epsilon
```
protected double m_epsilon
```
    The epsilon parameter for epsilon insensitive and Huber loss
  - m_t
```
protected double m_t
```
    Holds the current iteration number
  - m_numInstances
```
protected double m_numInstances
```
    The number of training instances
  - m_epochs
```
protected int m_epochs
```
    The number of epochs to perform (batch learning). Total iterations is m_epochs * num instances
  - m_dontNormalize
```
protected boolean m_dontNormalize
```
    Turn off normalization of the input data. This option gets forced for incremental training.
  - m_dontReplaceMissing
```
protected boolean m_dontReplaceMissing
```
    Turn off global replacement of missing values. Missing values will be ignored instead. This option gets forced for incremental training.
  - m_data
```
protected Instances m_data
```
    Holds the header of the training data
  - HINGE
```
public static final int HINGE
```
    the hinge loss function.
    
    See Also:
    
    Constant Field Values
  - LOGLOSS
```
public static final int LOGLOSS
```
    the log loss function.
    
    See Also:
    
    Constant Field Values
  - SQUAREDLOSS
```
public static final int SQUAREDLOSS
```
    the squared loss function.
    
    See Also:
    
    Constant Field Values
  - EPSILON_INSENSITIVE
```
public static final int EPSILON_INSENSITIVE
```
    The epsilon insensitive loss function
    
    See Also:
    
    Constant Field Values
  - HUBER
```
public static final int HUBER
```
    The Huber loss function
    
    See Also:
    
    Constant Field Values
  - m_loss
```
protected int m_loss
```
    The current loss function to minimize
  - TAGS_SELECTION
```
public static final Tag[] TAGS_SELECTION
```
    Loss functions to choose from
  - m_numModels
```
protected int m_numModels
```
- Constructor Detail
  - SGD
```
public SGD()
```
- Method Detail
  - getCapabilities
```
public Capabilities getCapabilities()
```
    Returns default capabilities of the classifier.
    
    Specified by:
    
    getCapabilities in interface Classifier
    
    Specified by:
    
    getCapabilities in interface CapabilitiesHandler
    
    Overrides:
    
    getCapabilities in class AbstractClassifier
    
    Returns:
    
    the capabilities of this classifier
    
    See Also:
    
    Capabilities
  - epsilonTipText
```
public java.lang.String epsilonTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setEpsilon
```
public void setEpsilon(double e)
```
    Set the epsilon threshold on the error for epsilon insensitive and Huber loss functions
    
    Parameters:
    
    e - the value of epsilon to use
  - getEpsilon
```
public double getEpsilon()
```
    Get the epsilon threshold on the error for epsilon insensitive and Huber loss functions
    
    Returns:
    
    the value of epsilon to use
  - lambdaTipText
```
public java.lang.String lambdaTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setLambda
```
public void setLambda(double lambda)
```
    Set the value of lambda to use
    
    Parameters:
    
    lambda - the value of lambda to use
  - getLambda
```
public double getLambda()
```
    Get the current value of lambda
    
    Returns:
    
    the current value of lambda
  - setLearningRate
```
public void setLearningRate(double lr)
```
    Set the learning rate.
    
    Parameters:
    
    lr - the learning rate to use.
  - getLearningRate
```
public double getLearningRate()
```
    Get the learning rate.
    
    Returns:
    
    the learning rate
  - learningRateTipText
```
public java.lang.String learningRateTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - epochsTipText
```
public java.lang.String epochsTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setEpochs
```
public void setEpochs(int e)
```
    Set the number of epochs to use
    
    Parameters:
    
    e - the number of epochs to use
  - getEpochs
```
public int getEpochs()
```
    Get current number of epochs
    
    Returns:
    
    the current number of epochs
  - setDontNormalize
```
public void setDontNormalize(boolean m)
```
    Turn normalization off/on.
    
    Parameters:
    
    m - true if normalization is to be disabled.
  - getDontNormalize
```
public boolean getDontNormalize()
```
    Get whether normalization has been turned off.
    
    Returns:
    
    true if normalization has been disabled.
  - dontNormalizeTipText
```
public java.lang.String dontNormalizeTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setDontReplaceMissing
```
public void setDontReplaceMissing(boolean m)
```
    Turn global replacement of missing values off/on. If turned off, then missing values are effectively ignored.
    
    Parameters:
    
    m - true if global replacement of missing values is to be turned off.
  - getDontReplaceMissing
```
public boolean getDontReplaceMissing()
```
    Get whether global replacement of missing values has been disabled.
    
    Returns:
    
    true if global replacement of missing values has been turned off
  - dontReplaceMissingTipText
```
public java.lang.String dontReplaceMissingTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setLossFunction
```
public void setLossFunction(SelectedTag function)
```
    Set the loss function to use.
    
    Parameters:
    
    function - the loss function to use.
  - getLossFunction
```
public SelectedTag getLossFunction()
```
    Get the current loss function.
    
    Returns:
    
    the current loss function.
  - lossFunctionTipText
```
public java.lang.String lossFunctionTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - listOptions
```
public java.util.Enumeration<Option> listOptions()
```
    Returns an enumeration describing the available options.
    
    Specified by:
    
    listOptions in interface OptionHandler
    
    Overrides:
    
    listOptions in class RandomizableClassifier
    
    Returns:
    
    an enumeration of all the available options.
  - setOptions
```
public void setOptions(java.lang.String[] options)
                throws java.lang.Exception
```
    Parses a given list of options.
    Valid options are:
```
  -F
  Set the loss function to minimize.
  0 = hinge loss (SVM), 1 = log loss (logistic regression),
  2 = squared loss (regression), 3 = epsilon insensitive loss (regression),
  4 = Huber loss (regression).
  (default = 0)
 
```
```
  -L
  The learning rate. If normalization is
  turned off (as it is automatically for streaming data), then the
  default learning rate will need to be reduced (try 0.0001).
  (default = 0.01).
 
```
```
  -R <double>
  The lambda regularization constant (default = 0.0001)
 
```
```
  -E <integer>
  The number of epochs to perform (batch learning only, default = 500)
 
```
```
  -C <double>
  The epsilon threshold (epsilon-insenstive and Huber loss only, default = 1e-3)
 
```
```
  -N
  Don't normalize the data
 
```
```
  -M
  Don't replace missing values
 
```
```
  -S <num>
  Random number seed.
  (default 1)
 
```
```
  -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console
 
```
```
  -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).
 
```
    Specified by:
    
    setOptions in interface OptionHandler
    
    Overrides:
    
    setOptions in class RandomizableClassifier
    
    Parameters:
    
    options - the list of options as an array of strings
    
    Throws:
    
    java.lang.Exception - if an option is not supported
  - getOptions
```
public java.lang.String[] getOptions()
```
    Gets the current settings of the classifier.
    
    Specified by:
    
    getOptions in interface OptionHandler
    
    Overrides:
    
    getOptions in class RandomizableClassifier
    
    Returns:
    
    an array of strings suitable for passing to setOptions
  - globalInfo
```
public java.lang.String globalInfo()
```
    Returns a string describing classifier
    
    Returns:
    
    a description suitable for displaying in the explorer/experimenter gui
  - reset
```
public void reset()
```
    Reset the classifier.
  - buildClassifier
```
public void buildClassifier(Instances data)
                     throws java.lang.Exception
```
    Method for building the classifier.
    
    Specified by:
    
    buildClassifier in interface Classifier
    
    Parameters:
    
    data - the set of training instances.
    
    Throws:
    
    java.lang.Exception - if the classifier can't be built successfully.
  - dloss
```
protected double dloss(double z)
```
  - dotProd
```
protected static double dotProd(Instance inst1,
                                double[] weights,
                                int classIndex)
                         throws java.lang.InterruptedException
```
    Throws:
    
    java.lang.InterruptedException
  - updateClassifier
```
protected void updateClassifier(Instance instance,
                                boolean filter)
                         throws java.lang.Exception
```
    Updates the classifier with the given instance.
    
    Parameters:
    
    instance - the new training instance to include in the model
    
    filter - true if the instance should pass through any of the filters set up in buildClassifier(). When batch training buildClassifier() already batch filters all training instances so don't need to filter them again here.
    
    Throws:
    
    java.lang.Exception - if the instance could not be incorporated in the model.
  - updateClassifier
```
public void updateClassifier(Instance instance)
                      throws java.lang.Exception
```
    Updates the classifier with the given instance.
    
    Specified by:
    
    updateClassifier in interface UpdateableClassifier
    
    Parameters:
    
    instance - the new training instance to include in the model
    
    Throws:
    
    java.lang.Exception - if the instance could not be incorporated in the model.
  - distributionForInstance
```
public double[] distributionForInstance(Instance inst)
                                 throws java.lang.Exception
```
    Computes the distribution for a given instance
    
    Specified by:
    
    distributionForInstance in interface Classifier
    
    Overrides:
    
    distributionForInstance in class AbstractClassifier
    
    Parameters:
    
    instance - the instance for which distribution is computed
    
    Returns:
    
    the distribution
    
    Throws:
    
    java.lang.Exception - if the distribution can't be computed successfully
  - getWeights
```
public double[] getWeights()
```
  - toString
```
public java.lang.String toString()
```
    Prints out the classifier.
    
    Overrides:
    
    toString in class java.lang.Object
    
    Returns:
    
    a description of the classifier as a string
  - getRevision
```
public java.lang.String getRevision()
```
    Returns the revision string.
    
    Specified by:
    
    getRevision in interface RevisionHandler
    
    Overrides:
    
    getRevision in class AbstractClassifier
    
    Returns:
    
    the revision
  - aggregate
```
public SGD aggregate(SGD toAggregate)
              throws java.lang.Exception
```
    Aggregate an object with this one
    
    Specified by:
    
    aggregate in interface Aggregateable<SGD>
    
    Parameters:
    
    toAggregate - the object to aggregate
    
    Returns:
    
    the result of aggregation
    
    Throws:
    
    java.lang.Exception - if the supplied object can't be aggregated for some reason
  - finalizeAggregation
```
public void finalizeAggregation()
                         throws java.lang.Exception
```
    Call to complete the aggregation process. Allows implementers to do any final processing based on how many objects were aggregated.
    
    Specified by:
    
    finalizeAggregation in interface Aggregateable<SGD>
    
    Throws:
    
    java.lang.Exception - if the aggregation can't be finalized for some reason
  - main
```
public static void main(java.lang.String[] args)
```
    Main method for testing this class.

Class SGD

Field Summary

Fields inherited from class weka.classifiers.RandomizableClassifier

Fields inherited from class weka.classifiers.AbstractClassifier

Constructor Summary

Method Summary

Methods inherited from class weka.classifiers.RandomizableClassifier

Methods inherited from class weka.classifiers.AbstractClassifier

Methods inherited from class java.lang.Object

Field Detail

m_replaceMissing

m_nominalToBinary

m_normalize

m_lambda

m_learningRate

m_weights

m_epsilon

m_t

m_numInstances

m_epochs

m_dontNormalize

m_dontReplaceMissing

m_data

HINGE

LOGLOSS

SQUAREDLOSS

EPSILON_INSENSITIVE

HUBER

m_loss

TAGS_SELECTION

m_numModels

Constructor Detail

SGD

Method Detail

getCapabilities

epsilonTipText

setEpsilon

getEpsilon

lambdaTipText

setLambda

getLambda

setLearningRate

getLearningRate

learningRateTipText

epochsTipText

setEpochs

getEpochs

setDontNormalize

getDontNormalize

dontNormalizeTipText

setDontReplaceMissing

getDontReplaceMissing

dontReplaceMissingTipText

setLossFunction

getLossFunction

lossFunctionTipText

listOptions

setOptions

getOptions

globalInfo

reset

buildClassifier

dloss

dotProd

updateClassifier

updateClassifier

distributionForInstance

getWeights

toString

getRevision

aggregate

finalizeAggregation

main