SGDText (interruptible-weka 0.1.4 API)

java.lang.Object
- weka.classifiers.AbstractClassifier
- - weka.classifiers.RandomizableClassifier
  - - weka.classifiers.functions.SGDText

All Implemented Interfaces:: java.io.Serializable, java.lang.Cloneable, Classifier, UpdateableBatchProcessor, UpdateableClassifier, Aggregateable<SGDText>, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, Randomizable, RevisionHandler, WeightedInstancesHandler

public class SGDText
extends RandomizableClassifier
implements UpdateableClassifier, UpdateableBatchProcessor, WeightedInstancesHandler, Aggregateable<SGDText>

Implements stochastic gradient descent for learning a linear binary class SVM or binary class logistic regression on text data. Operates directly (and only) on String attributes. Other types of input attributes are accepted but ignored during training and classification.

Valid options are:

 -F
  Set the loss function to minimize. 0 = hinge loss (SVM), 1 = log loss (logistic regression)
  (default = 0)

 -outputProbs
  Output probabilities for SVMs (fits a logsitic
  model to the output of the SVM)

 -L
  The learning rate (default = 0.01).

 -R <double>
  The lambda regularization constant (default = 0.0001)

 -E <integer>
  The number of epochs to perform (batch learning only, default = 500)

 -W
  Use word frequencies instead of binary bag of words.

 -P <# instances>
  How often to prune the dictionary of low frequency words (default = 0, i.e. don't prune)

 -M <double>
  Minimum word frequency. Words with less than this frequence are ignored.
  If periodic pruning is turned on then this is also used to determine which
  words to remove from the dictionary (default = 3).

 -min-coeff <double>
  Minimum absolute value of coefficients in the model.
  If periodic pruning is turned on then this
  is also used to prune words from the dictionary
  (default = 0.001

 -normalize
  Normalize document length (use in conjunction with -norm and -lnorm)

 -norm <num>
  Specify the norm that each instance must have (default 1.0)

 -lnorm <num>
  Specify L-norm to use (default 2.0)

 -lowercase
  Convert all tokens to lowercase before adding to the dictionary.

 -stopwords-handler
  The stopwords handler to use (default Null).

 -tokenizer <spec>
  The tokenizing algorihtm (classname plus parameters) to use.
  (default: weka.core.tokenizers.WordTokenizer)

 -stemmer <spec>
  The stemmering algorihtm (classname plus parameters) to use.

 -S <num>
  Random number seed.
  (default 1)

 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console

 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).

See Also:: Serialized Form

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class SGDText.Count

Nested Classes
Modifier and Type	Class and Description
`static class`	`SGDText.Count`

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`HINGE` the hinge loss function.
`static int`	`LOGLOSS` the log loss function.
`protected double`	`m_bias` Holds the bias term
`protected Instances`	`m_data` The header of the training data
`protected java.util.LinkedHashMap<java.lang.String,SGDText.Count>`	`m_dictionary` The dictionary (and term weights)
`protected int`	`m_epochs` The number of epochs to perform (batch learning).
`protected boolean`	`m_fitLogistic` True if a logistic regression is to be fit to the output of the SVM for producing probability estimates
`protected Instances`	`m_fitLogisticStructure`
`protected java.util.LinkedHashMap<java.lang.String,SGDText.Count>`	`m_inputVector` Holds the current document vector (LinkedHashMap is more efficient when iterating over EntrySet than HashMap)
`protected double`	`m_lambda` The regularization parameter
`protected double`	`m_learningRate` The learning rate
`protected double`	`m_lnorm` The L-norm to use
`protected int`	`m_loss` The current loss function to minimize
`protected boolean`	`m_lowercaseTokens` Whether or not to convert all tokens to lowercase
`protected double`	`m_minAbsCoefficient` Prune terms from the model that have a coefficient smaller than this.
`protected double`	`m_minWordP` Only consider dictionary words (features) that occur at least this many times.
`protected double`	`m_norm` The length that each document vector should have in the end
`protected boolean`	`m_normalize` Whether to normalized document length or not
`protected double`	`m_numInstances` The number of training instances
`protected int`	`m_numModels`
`protected int`	`m_periodicP` The number of training instances at which to periodically prune the dictionary of min frequency words.
`protected Stemmer`	`m_stemmer` The stemming algorithm.
`protected StopwordsHandler`	`m_StopwordsHandler` Stopword handler to use.
`protected SGD`	`m_svmProbs` Used for producing probabilities for SVM via SGD logistic regression
`protected double`	`m_t` Holds the current iteration number
`protected Tokenizer`	`m_tokenizer` The tokenizer to use
`protected boolean`	`m_wordFrequencies` Use word frequencies rather than bag-of-words if true
`static Tag[]`	`TAGS_SELECTION` Loss functions to choose from

Fields inherited from class weka.classifiers.RandomizableClassifier
m_Seed

Fields inherited from class weka.classifiers.AbstractClassifier
BATCH_SIZE_DEFAULT, m_BatchSize, m_Debug, m_DoNotCheckCapabilities, m_numDecimalPlaces, NUM_DECIMAL_PLACES_DEFAULT

Constructor Summary

Constructors
Constructor and Description

SGDText()

Constructors
Constructor and Description
`SGDText()`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`SGDText`	`aggregate(SGDText toAggregate)` Aggregate an object with this one
`void`	`batchFinished()` Signal that the training data is finished (for now).
`double`	`bias()`
`void`	`buildClassifier(Instances data)` Method for building the classifier.
`double[]`	`distributionForInstance(Instance inst)` Predicts the class memberships for a given instance.
`protected double`	`dloss(double z)`
`protected double`	`dotProd(java.util.Map<java.lang.String,SGDText.Count> document)`
`java.lang.String`	`epochsTipText()` Returns the tip text for this property
`void`	`finalizeAggregation()` Call to complete the aggregation process.
`Capabilities`	`getCapabilities()` Returns default capabilities of the classifier.
`java.util.LinkedHashMap<java.lang.String,SGDText.Count>`	`getDictionary()` Get this model's dictionary (including term weights).
`int`	`getDictionarySize()` Return the size of the dictionary (minus any low frequency terms that are below the threshold but haven't been pruned yet).
`int`	`getEpochs()` Get current number of epochs
`double`	`getLambda()` Get the current value of lambda
`double`	`getLearningRate()` Get the learning rate.
`double`	`getLNorm()` Get the L Norm used.
`SelectedTag`	`getLossFunction()` Get the current loss function.
`boolean`	`getLowercaseTokens()` Get whether to convert all tokens to lowercase
`double`	`getMinAbsoluteCoefficientValue()` Get the minimum absolute magnitude for model coefficients.
`double`	`getMinWordFrequency()` Get the minimum word frequency.
`double`	`getNorm()` Get the instance's Norm.
`boolean`	`getNormalizeDocLength()` Get whether to normalize the length of each document
`java.lang.String[]`	`getOptions()` Gets the current settings of the classifier.
`boolean`	`getOutputProbsForSVM()` Get whether to fit a logistic regression (itself trained using SGD) to the outputs of the SVM (if an SVM is being learned).
`int`	`getPeriodicPruning()` Get how often to prune the dictionary
`java.lang.String`	`getRevision()` Returns the revision string.
`Stemmer`	`getStemmer()` Returns the current stemming algorithm, null if none is used.
`StopwordsHandler`	`getStopwordsHandler()` Gets the stopwords handler.
`Tokenizer`	`getTokenizer()` Returns the current tokenizer algorithm.
`boolean`	`getUseWordFrequencies()` Get whether to use word frequencies rather than binary bag of words representation.
`java.lang.String`	`globalInfo()` Returns a string describing classifier
`protected void`	`initializeSVMProbs(Instances data)`
`java.lang.String`	`lambdaTipText()` Returns the tip text for this property
`java.lang.String`	`learningRateTipText()` Returns the tip text for this property
`java.util.Enumeration<Option>`	`listOptions()` Returns an enumeration describing the available options.
`java.lang.String`	`LNormTipText()` Returns the tip text for this property
`java.lang.String`	`lossFunctionTipText()` Returns the tip text for this property
`java.lang.String`	`lowercaseTokensTipText()` Returns the tip text for this property
`static void`	`main(java.lang.String[] args)` Main method for testing this class.
`java.lang.String`	`minAbsoluteCoefficientValueTipText()` Returns the tip text for this property
`java.lang.String`	`minWordFrequencyTipText()` Returns the tip text for this property
`java.lang.String`	`normalizeDocLengthTipText()` Returns the tip text for this property
`java.lang.String`	`normTipText()` Returns the tip text for this property
`java.lang.String`	`outputProbsForSVMTipText()` Returns the tip text for this property
`java.lang.String`	`periodicPruningTipText()` Returns the tip text for this property
`protected void`	`pruneDictionary(boolean force)`
`void`	`reset()` Reset the classifier.
`void`	`setBias(double bias)`
`void`	`setEpochs(int e)` Set the number of epochs to use
`void`	`setLambda(double lambda)` Set the value of lambda to use
`void`	`setLearningRate(double lr)` Set the learning rate.
`void`	`setLNorm(double newLNorm)` Set the L-norm to used
`void`	`setLossFunction(SelectedTag function)` Set the loss function to use.
`void`	`setLowercaseTokens(boolean l)` Set whether to convert all tokens to lowercase
`void`	`setMinAbsoluteCoefficientValue(double minCoeff)` Set the minimum absolute magnitude for model coefficients.
`void`	`setMinWordFrequency(double minFreq)` Set the minimum word frequency.
`void`	`setNorm(double newNorm)` Set the norm of the instances
`void`	`setNormalizeDocLength(boolean norm)` Set whether to normalize the length of each document
`void`	`setOptions(java.lang.String[] options)` Parses a given list of options.
`void`	`setOutputProbsForSVM(boolean o)` Set whether to fit a logistic regression (itself trained using SGD) to the outputs of the SVM (if an SVM is being learned).
`void`	`setPeriodicPruning(int p)` Set how often to prune the dictionary
`void`	`setStemmer(Stemmer value)` the stemming algorithm to use, null means no stemming at all (i.e., the NullStemmer is used).
`void`	`setStopwordsHandler(StopwordsHandler value)` Sets the stopwords handler to use.
`void`	`setTokenizer(Tokenizer value)` the tokenizer algorithm to use.
`void`	`setUseWordFrequencies(boolean u)` Set whether to use word frequencies rather than binary bag of words representation.
`java.lang.String`	`stemmerTipText()` Returns the tip text for this property.
`java.lang.String`	`stopwordsHandlerTipText()` Returns the tip text for this property.
`protected double`	`svmOutput()`
`protected void`	`tokenizeInstance(Instance instance, boolean updateDictionary)`
`java.lang.String`	`tokenizerTipText()` Returns the tip text for this property.
`java.lang.String`	`toString()`
`protected void`	`train(Instances data)`
`void`	`updateClassifier(Instance instance)` Updates the classifier with the given instance.
`protected void`	`updateClassifier(Instance instance, boolean updateDictionary)`
`java.lang.String`	`useWordFrequenciesTipText()` Returns the tip text for this property

Methods inherited from class weka.classifiers.RandomizableClassifier
getSeed, seedTipText, setSeed

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - m_periodicP
```
protected int m_periodicP
```
    The number of training instances at which to periodically prune the dictionary of min frequency words. Empty or null string indicates don't prune
  - m_minWordP
```
protected double m_minWordP
```
    Only consider dictionary words (features) that occur at least this many times.
  - m_minAbsCoefficient
```
protected double m_minAbsCoefficient
```
    Prune terms from the model that have a coefficient smaller than this.
  - m_wordFrequencies
```
protected boolean m_wordFrequencies
```
    Use word frequencies rather than bag-of-words if true
  - m_normalize
```
protected boolean m_normalize
```
    Whether to normalized document length or not
  - m_norm
```
protected double m_norm
```
    The length that each document vector should have in the end
  - m_lnorm
```
protected double m_lnorm
```
    The L-norm to use
  - m_dictionary
```
protected java.util.LinkedHashMap<java.lang.String,SGDText.Count> m_dictionary
```
    The dictionary (and term weights)
  - m_StopwordsHandler
```
protected StopwordsHandler m_StopwordsHandler
```
    Stopword handler to use.
  - m_tokenizer
```
protected Tokenizer m_tokenizer
```
    The tokenizer to use
  - m_lowercaseTokens
```
protected boolean m_lowercaseTokens
```
    Whether or not to convert all tokens to lowercase
  - m_stemmer
```
protected Stemmer m_stemmer
```
    The stemming algorithm.
  - m_lambda
```
protected double m_lambda
```
    The regularization parameter
  - m_learningRate
```
protected double m_learningRate
```
    The learning rate
  - m_t
```
protected double m_t
```
    Holds the current iteration number
  - m_bias
```
protected double m_bias
```
    Holds the bias term
  - m_numInstances
```
protected double m_numInstances
```
    The number of training instances
  - m_data
```
protected Instances m_data
```
    The header of the training data
  - m_epochs
```
protected int m_epochs
```
    The number of epochs to perform (batch learning). Total iterations is m_epochs * num instances
  - m_inputVector
```
protected transient java.util.LinkedHashMap<java.lang.String,SGDText.Count> m_inputVector
```
    Holds the current document vector (LinkedHashMap is more efficient when iterating over EntrySet than HashMap)
  - HINGE
```
public static final int HINGE
```
    the hinge loss function.
    
    See Also:
    
    Constant Field Values
  - LOGLOSS
```
public static final int LOGLOSS
```
    the log loss function.
    
    See Also:
    
    Constant Field Values
  - m_loss
```
protected int m_loss
```
    The current loss function to minimize
  - TAGS_SELECTION
```
public static final Tag[] TAGS_SELECTION
```
    Loss functions to choose from
  - m_svmProbs
```
protected SGD m_svmProbs
```
    Used for producing probabilities for SVM via SGD logistic regression
  - m_fitLogistic
```
protected boolean m_fitLogistic
```
    True if a logistic regression is to be fit to the output of the SVM for producing probability estimates
  - m_fitLogisticStructure
```
protected Instances m_fitLogisticStructure
```
  - m_numModels
```
protected int m_numModels
```
- Constructor Detail
  - SGDText
```
public SGDText()
```
- Method Detail
  - dloss
```
protected double dloss(double z)
```
  - getCapabilities
```
public Capabilities getCapabilities()
```
    Returns default capabilities of the classifier.
    
    Specified by:
    
    getCapabilities in interface Classifier
    
    Specified by:
    
    getCapabilities in interface CapabilitiesHandler
    
    Overrides:
    
    getCapabilities in class AbstractClassifier
    
    Returns:
    
    the capabilities of this classifier
    
    See Also:
    
    Capabilities
  - setStemmer
```
public void setStemmer(Stemmer value)
```
    the stemming algorithm to use, null means no stemming at all (i.e., the NullStemmer is used).
    
    Parameters:
    
    value - the configured stemming algorithm, or null
    
    See Also:
    
    NullStemmer
  - getStemmer
```
public Stemmer getStemmer()
```
    Returns the current stemming algorithm, null if none is used.
    
    Returns:
    
    the current stemming algorithm, null if none set
  - stemmerTipText
```
public java.lang.String stemmerTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setTokenizer
```
public void setTokenizer(Tokenizer value)
```
    the tokenizer algorithm to use.
    
    Parameters:
    
    value - the configured tokenizing algorithm
  - getTokenizer
```
public Tokenizer getTokenizer()
```
    Returns the current tokenizer algorithm.
    
    Returns:
    
    the current tokenizer algorithm
  - tokenizerTipText
```
public java.lang.String tokenizerTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - useWordFrequenciesTipText
```
public java.lang.String useWordFrequenciesTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setUseWordFrequencies
```
public void setUseWordFrequencies(boolean u)
```
    Set whether to use word frequencies rather than binary bag of words representation.
    
    Parameters:
    
    u - true if word frequencies are to be used.
  - getUseWordFrequencies
```
public boolean getUseWordFrequencies()
```
    Get whether to use word frequencies rather than binary bag of words representation.
    
    Returns:
    
    true if word frequencies are to be used.
  - lowercaseTokensTipText
```
public java.lang.String lowercaseTokensTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setLowercaseTokens
```
public void setLowercaseTokens(boolean l)
```
    Set whether to convert all tokens to lowercase
    
    Parameters:
    
    l - true if all tokens are to be converted to lowercase
  - getLowercaseTokens
```
public boolean getLowercaseTokens()
```
    Get whether to convert all tokens to lowercase
    
    Returns:
    
    true true if all tokens are to be converted to lowercase
  - setStopwordsHandler
```
public void setStopwordsHandler(StopwordsHandler value)
```
    Sets the stopwords handler to use.
    
    Parameters:
    
    value - the stopwords handler, if null, Null is used
  - getStopwordsHandler
```
public StopwordsHandler getStopwordsHandler()
```
    Gets the stopwords handler.
    
    Returns:
    
    the stopwords handler
  - stopwordsHandlerTipText
```
public java.lang.String stopwordsHandlerTipText()
```
    Returns the tip text for this property.
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - periodicPruningTipText
```
public java.lang.String periodicPruningTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setPeriodicPruning
```
public void setPeriodicPruning(int p)
```
    Set how often to prune the dictionary
    
    Parameters:
    
    p - how often to prune
  - getPeriodicPruning
```
public int getPeriodicPruning()
```
    Get how often to prune the dictionary
    
    Returns:
    
    how often to prune the dictionary
  - minWordFrequencyTipText
```
public java.lang.String minWordFrequencyTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setMinWordFrequency
```
public void setMinWordFrequency(double minFreq)
```
    Set the minimum word frequency. Words that don't occur at least min freq times are ignored when updating weights. If periodic pruning is turned on, then min frequency is used when removing words from the dictionary.
    
    Parameters:
    
    minFreq - the minimum word frequency to use
  - getMinWordFrequency
```
public double getMinWordFrequency()
```
    Get the minimum word frequency. Words that don't occur at least min freq times are ignored when updating weights. If periodic pruning is turned on, then min frequency is used when removing words from the dictionary.
    
    Returns:
    
    the minimum word frequency to use
  - minAbsoluteCoefficientValueTipText
```
public java.lang.String minAbsoluteCoefficientValueTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setMinAbsoluteCoefficientValue
```
public void setMinAbsoluteCoefficientValue(double minCoeff)
```
    Set the minimum absolute magnitude for model coefficients. Terms with weights smaller than this value are ignored. If periodic pruning is turned on then this is also used to determine if a word should be removed from the dictionary
    
    Parameters:
    
    minCoeff - the minimum absolute value of a model coefficient
  - getMinAbsoluteCoefficientValue
```
public double getMinAbsoluteCoefficientValue()
```
    Get the minimum absolute magnitude for model coefficients. Terms with weights smaller than this value are ignored. If periodic pruning is turned on this then is also used to determine if a word should be removed from the dictionary
    
    Returns:
    
    the minimum absolute value of a model coefficient
  - normalizeDocLengthTipText
```
public java.lang.String normalizeDocLengthTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setNormalizeDocLength
```
public void setNormalizeDocLength(boolean norm)
```
    Set whether to normalize the length of each document
    
    Parameters:
    
    norm - true if document lengths is to be normalized
  - getNormalizeDocLength
```
public boolean getNormalizeDocLength()
```
    Get whether to normalize the length of each document
    
    Returns:
    
    true if document lengths is to be normalized
  - normTipText
```
public java.lang.String normTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getNorm
```
public double getNorm()
```
    Get the instance's Norm.
    
    Returns:
    
    the Norm
  - setNorm
```
public void setNorm(double newNorm)
```
    Set the norm of the instances
    
    Parameters:
    
    newNorm - the norm to wich the instances must be set
  - LNormTipText
```
public java.lang.String LNormTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - getLNorm
```
public double getLNorm()
```
    Get the L Norm used.
    
    Returns:
    
    the L-norm used
  - setLNorm
```
public void setLNorm(double newLNorm)
```
    Set the L-norm to used
    
    Parameters:
    
    newLNorm - the L-norm
  - lambdaTipText
```
public java.lang.String lambdaTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setLambda
```
public void setLambda(double lambda)
```
    Set the value of lambda to use
    
    Parameters:
    
    lambda - the value of lambda to use
  - getLambda
```
public double getLambda()
```
    Get the current value of lambda
    
    Returns:
    
    the current value of lambda
  - setLearningRate
```
public void setLearningRate(double lr)
```
    Set the learning rate.
    
    Parameters:
    
    lr - the learning rate to use.
  - getLearningRate
```
public double getLearningRate()
```
    Get the learning rate.
    
    Returns:
    
    the learning rate
  - learningRateTipText
```
public java.lang.String learningRateTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - epochsTipText
```
public java.lang.String epochsTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setEpochs
```
public void setEpochs(int e)
```
    Set the number of epochs to use
    
    Parameters:
    
    e - the number of epochs to use
  - getEpochs
```
public int getEpochs()
```
    Get current number of epochs
    
    Returns:
    
    the current number of epochs
  - setLossFunction
```
public void setLossFunction(SelectedTag function)
```
    Set the loss function to use.
    
    Parameters:
    
    function - the loss function to use.
  - getLossFunction
```
public SelectedTag getLossFunction()
```
    Get the current loss function.
    
    Returns:
    
    the current loss function.
  - lossFunctionTipText
```
public java.lang.String lossFunctionTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - setOutputProbsForSVM
```
public void setOutputProbsForSVM(boolean o)
```
    Set whether to fit a logistic regression (itself trained using SGD) to the outputs of the SVM (if an SVM is being learned).
    
    Parameters:
    
    o - true if a logistic regression is to be fit to the output of the SVM to produce probability estimates.
  - getOutputProbsForSVM
```
public boolean getOutputProbsForSVM()
```
    Get whether to fit a logistic regression (itself trained using SGD) to the outputs of the SVM (if an SVM is being learned).
    
    Returns:
    
    true if a logistic regression is to be fit to the output of the SVM to produce probability estimates.
  - outputProbsForSVMTipText
```
public java.lang.String outputProbsForSVMTipText()
```
    Returns the tip text for this property
    
    Returns:
    
    tip text for this property suitable for displaying in the explorer/experimenter gui
  - listOptions
```
public java.util.Enumeration<Option> listOptions()
```
    Returns an enumeration describing the available options.
    
    Specified by:
    
    listOptions in interface OptionHandler
    
    Overrides:
    
    listOptions in class RandomizableClassifier
    
    Returns:
    
    an enumeration of all the available options.
  - setOptions
```
public void setOptions(java.lang.String[] options)
                throws java.lang.Exception
```
    Parses a given list of options.
    Valid options are:
```
 -F
  Set the loss function to minimize. 0 = hinge loss (SVM), 1 = log loss (logistic regression)
  (default = 0)
```
```
 -outputProbs
  Output probabilities for SVMs (fits a logsitic
  model to the output of the SVM)
```
```
 -L
  The learning rate (default = 0.01).
```
```
 -R <double>
  The lambda regularization constant (default = 0.0001)
```
```
 -E <integer>
  The number of epochs to perform (batch learning only, default = 500)
```
```
 -W
  Use word frequencies instead of binary bag of words.
```
```
 -P <# instances>
  How often to prune the dictionary of low frequency words (default = 0, i.e. don't prune)
```
```
 -M <double>
  Minimum word frequency. Words with less than this frequence are ignored.
  If periodic pruning is turned on then this is also used to determine which
  words to remove from the dictionary (default = 3).
```
```
 -min-coeff <double>
  Minimum absolute value of coefficients in the model.
  If periodic pruning is turned on then this
  is also used to prune words from the dictionary
  (default = 0.001
```
```
 -normalize
  Normalize document length (use in conjunction with -norm and -lnorm)
```
```
 -norm <num>
  Specify the norm that each instance must have (default 1.0)
```
```
 -lnorm <num>
  Specify L-norm to use (default 2.0)
```
```
 -lowercase
  Convert all tokens to lowercase before adding to the dictionary.
```
```
 -stopwords-handler
  The stopwords handler to use (default Null).
```
```
 -tokenizer <spec>
  The tokenizing algorihtm (classname plus parameters) to use.
  (default: weka.core.tokenizers.WordTokenizer)
```
```
 -stemmer <spec>
  The stemmering algorihtm (classname plus parameters) to use.
```
```
 -S <num>
  Random number seed.
  (default 1)
```
```
 -output-debug-info
  If set, classifier is run in debug mode and
  may output additional info to the console
```
```
 -do-not-check-capabilities
  If set, classifier capabilities are not checked before classifier is built
  (use with caution).
```
    Specified by:
    
    setOptions in interface OptionHandler
    
    Overrides:
    
    setOptions in class RandomizableClassifier
    
    Parameters:
    
    options - the list of options as an array of strings
    
    Throws:
    
    java.lang.Exception - if an option is not supported
  - getOptions
```
public java.lang.String[] getOptions()
```
    Gets the current settings of the classifier.
    
    Specified by:
    
    getOptions in interface OptionHandler
    
    Overrides:
    
    getOptions in class RandomizableClassifier
    
    Returns:
    
    an array of strings suitable for passing to setOptions
  - globalInfo
```
public java.lang.String globalInfo()
```
    Returns a string describing classifier
    
    Returns:
    
    a description suitable for displaying in the explorer/experimenter gui
  - reset
```
public void reset()
```
    Reset the classifier.
  - buildClassifier
```
public void buildClassifier(Instances data)
                     throws java.lang.Exception
```
    Method for building the classifier.
    
    Specified by:
    
    buildClassifier in interface Classifier
    
    Parameters:
    
    data - the set of training instances.
    
    Throws:
    
    java.lang.Exception - if the classifier can't be built successfully.
  - initializeSVMProbs
```
protected void initializeSVMProbs(Instances data)
                           throws java.lang.Exception
```
    Throws:
    
    java.lang.Exception
  - train
```
protected void train(Instances data)
              throws java.lang.Exception
```
    Throws:
    
    java.lang.Exception
  - updateClassifier
```
public void updateClassifier(Instance instance)
                      throws java.lang.Exception
```
    Updates the classifier with the given instance.
    
    Specified by:
    
    updateClassifier in interface UpdateableClassifier
    
    Parameters:
    
    instance - the new training instance to include in the model
    
    Throws:
    
    java.lang.Exception - if the instance could not be incorporated in the model.
  - updateClassifier
```
protected void updateClassifier(Instance instance,
                                boolean updateDictionary)
                         throws java.lang.Exception
```
    Throws:
    
    java.lang.Exception
  - tokenizeInstance
```
protected void tokenizeInstance(Instance instance,
                                boolean updateDictionary)
```
  - pruneDictionary
```
protected void pruneDictionary(boolean force)
```
  - svmOutput
```
protected double svmOutput()
```
  - distributionForInstance
```
public double[] distributionForInstance(Instance inst)
                                 throws java.lang.Exception
```
    Description copied from class: AbstractClassifier
    
    Predicts the class memberships for a given instance. If an instance is unclassified, the returned array elements must be all zero. If the class is numeric, the array must consist of only one element, which contains the predicted value. Note that a classifier MUST implement either this or classifyInstance().
    
    Specified by:
    
    distributionForInstance in interface Classifier
    
    Overrides:
    
    distributionForInstance in class AbstractClassifier
    
    Parameters:
    
    inst - the instance to be classified
    
    Returns:
    
    an array containing the estimated membership probabilities of the test instance in each class or the numeric prediction
    
    Throws:
    
    java.lang.Exception - if distribution could not be computed successfully
  - dotProd
```
protected double dotProd(java.util.Map<java.lang.String,SGDText.Count> document)
```
  - toString
```
public java.lang.String toString()
```
    Overrides:
    
    toString in class java.lang.Object
  - getDictionary
```
public java.util.LinkedHashMap<java.lang.String,SGDText.Count> getDictionary()
```
    Get this model's dictionary (including term weights).
    
    Returns:
    
    this model's dictionary.
  - getDictionarySize
```
public int getDictionarySize()
```
    Return the size of the dictionary (minus any low frequency terms that are below the threshold but haven't been pruned yet).
    
    Returns:
    
    the size of the dictionary.
  - bias
```
public double bias()
```
  - setBias
```
public void setBias(double bias)
```
  - getRevision
```
public java.lang.String getRevision()
```
    Returns the revision string.
    
    Specified by:
    
    getRevision in interface RevisionHandler
    
    Overrides:
    
    getRevision in class AbstractClassifier
    
    Returns:
    
    the revision
  - aggregate
```
public SGDText aggregate(SGDText toAggregate)
                  throws java.lang.Exception
```
    Aggregate an object with this one
    
    Specified by:
    
    aggregate in interface Aggregateable<SGDText>
    
    Parameters:
    
    toAggregate - the object to aggregate
    
    Returns:
    
    the result of aggregation
    
    Throws:
    
    java.lang.Exception - if the supplied object can't be aggregated for some reason
  - finalizeAggregation
```
public void finalizeAggregation()
                         throws java.lang.Exception
```
    Call to complete the aggregation process. Allows implementers to do any final processing based on how many objects were aggregated.
    
    Specified by:
    
    finalizeAggregation in interface Aggregateable<SGDText>
    
    Throws:
    
    java.lang.Exception - if the aggregation can't be finalized for some reason
  - batchFinished
```
public void batchFinished()
                   throws java.lang.Exception
```
    Description copied from interface: UpdateableBatchProcessor
    
    Signal that the training data is finished (for now).
    
    Specified by:
    
    batchFinished in interface UpdateableBatchProcessor
    
    Throws:
    
    java.lang.Exception - if a problem occurs
  - main
```
public static void main(java.lang.String[] args)
```
    Main method for testing this class.

Class SGDText

Nested Class Summary

Field Summary

Fields inherited from class weka.classifiers.RandomizableClassifier

Fields inherited from class weka.classifiers.AbstractClassifier

Constructor Summary

Method Summary

Methods inherited from class weka.classifiers.RandomizableClassifier

Methods inherited from class weka.classifiers.AbstractClassifier

Methods inherited from class java.lang.Object

Field Detail

m_periodicP

m_minWordP

m_minAbsCoefficient

m_wordFrequencies

m_normalize

m_norm

m_lnorm

m_dictionary

m_StopwordsHandler

m_tokenizer

m_lowercaseTokens

m_stemmer

m_lambda

m_learningRate

m_t

m_bias

m_numInstances

m_data

m_epochs

m_inputVector

HINGE

LOGLOSS

m_loss

TAGS_SELECTION

m_svmProbs

m_fitLogistic

m_fitLogisticStructure

m_numModels

Constructor Detail

SGDText

Method Detail

dloss

getCapabilities

setStemmer

getStemmer

stemmerTipText

setTokenizer

getTokenizer

tokenizerTipText

useWordFrequenciesTipText

setUseWordFrequencies

getUseWordFrequencies

lowercaseTokensTipText

setLowercaseTokens

getLowercaseTokens

setStopwordsHandler

getStopwordsHandler

stopwordsHandlerTipText

periodicPruningTipText

setPeriodicPruning

getPeriodicPruning

minWordFrequencyTipText

setMinWordFrequency

getMinWordFrequency

minAbsoluteCoefficientValueTipText

setMinAbsoluteCoefficientValue

getMinAbsoluteCoefficientValue

normalizeDocLengthTipText

setNormalizeDocLength

getNormalizeDocLength

normTipText

getNorm

setNorm

LNormTipText

getLNorm

setLNorm

lambdaTipText

setLambda

getLambda