LogisticBase (interruptible-weka 0.1.2 API)

java.lang.Object
- weka.classifiers.AbstractClassifier
- - weka.classifiers.trees.lmt.LogisticBase

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable, Classifier, BatchPredictor, CapabilitiesHandler, CapabilitiesIgnorer, CommandlineRunnable, OptionHandler, RevisionHandler, WeightedInstancesHandler

Direct Known Subclasses:

LMTNode
```
public class LogisticBase
extends AbstractClassifier
implements WeightedInstancesHandler
```
Base/helper class for building logistic regression models with the LogitBoost algorithm. Used for building logistic model trees (weka.classifiers.trees.lmt.LMT) and standalone logistic regression (weka.classifiers.functions.SimpleLogistic). Valid options are:
```
 -D
  If set, classifier is run in debug mode and
  may output additional info to the console
 
```
See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected boolean`	`m_errorOnProbabilities` Use error on probabilities for stopping criterion of LogitBoost?
`protected int`	`m_fixedNumIterations` Use fixed number of iterations for LogitBoost?
`protected int`	`m_heuristicStop` Use heuristic to stop performing LogitBoost iterations earlier?
`protected int`	`m_maxIterations` The maximum number of LogitBoost iterations
`protected int`	`m_numClasses` The number of different classes
`protected Instances`	`m_numericData` Numeric version of the training data.
`protected Instances`	`m_numericDataHeader` Header-only version of the numeric version of the training data
`protected static int`	`m_numFoldsBoosting` Number of folds for cross-validating number of LogitBoost iterations
`protected double`	`m_numParameters` Effective number of parameters used for AIC / BIC automatic stopping
`protected int`	`m_numRegressions` The number of LogitBoost iterations performed.
`protected SimpleLinearRegression[][]`	`m_regressions` Array holding the simple regression functions fit by LogitBoost
`protected Instances`	`m_train` Training data
`protected boolean`	`m_useCrossValidation` Use cross-validation to determine best number of LogitBoost iterations ?
`protected double`	`m_weightTrimBeta` Threshold for trimming weights.
`protected static double`	`Z_MAX` Threshold on the Z-value for LogitBoost

Fields inherited from class weka.classifiers.AbstractClassifier
BATCH_SIZE_DEFAULT, m_BatchSize, m_Debug, m_DoNotCheckCapabilities, m_numDecimalPlaces, NUM_DECIMAL_PLACES_DEFAULT

Constructor Summary

Constructors
Constructor and Description
`LogisticBase()` Constructor that creates LogisticBase object with standard options.
`LogisticBase(int numBoostingIterations, boolean useCrossValidation, boolean errorOnProbabilities)` Constructor to create LogisticBase object.

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`buildClassifier(Instances data)` Builds the logistic regression model usiing LogitBoost.
`void`	`cleanup()` Cleanup in order to save memory.
`protected SimpleLinearRegression[][]`	`copyRegressions(SimpleLinearRegression[][] a)` Deep copies the given array of simple linear regression functions.
`double[]`	`distributionForInstance(Instance instance)` Returns class probabilities for an instance.
`protected int`	`getBestIteration(double[] errors, int maxIteration)` Helper function to find the minimum in an array of error values.
`protected double[][]`	`getCoefficients()` Returns an array holding the coefficients of the logistic model.
`protected double`	`getErrorRate(Instances data)` Returns the misclassification error of the current model on a set of instances.
`protected double[]`	`getFs(Instance instance)` Computes the F-values for a single instance.
`protected double[][]`	`getFs(Instances data)` Computes the F-values for a set of instances.
`int`	`getMaxIterations()` Returns the maxIterations parameter.
`protected double`	`getMeanAbsoluteError(Instances data)` Returns the error of the probability estimates for the current model on a set of instances.
`protected Instances`	`getNumericData(Instances data)` Converts training data to numeric version.
`int`	`getNumRegressions()` The number of LogitBoost iterations performed (= the number of simple regression functions fit).
`protected double[][]`	`getProbs(double[][] dataFs)` Computes the p-values (probabilities for the different classes) from the F-values for a set of instances.
`java.lang.String`	`getRevision()` Returns the revision string.
`boolean`	`getUseAIC()` Get the value of useAIC.
`int[][]`	`getUsedAttributes()` Returns an array of the indices of the attributes used in the logistic model.
`double`	`getWeightTrimBeta()` Get the value of weightTrimBeta.
`protected double[][]`	`getWs(double[][] probs, double[][] dataYs)` Computes the LogitBoost weights from an array of y/p values (actual/estimated class probabilities).
`protected double[][]`	`getYs(Instances data)` Computes the Y-values (actual class probabilities) for a set of instances.
`protected double`	`getZ(double actual, double p)` Computes the LogitBoost response variable from y/p values (actual/estimated class probabilities).
`protected double[][]`	`getZs(double[][] probs, double[][] dataYs)` Computes the LogitBoost response for an array of y/p values (actual/estimated class probabilities).
`protected SimpleLinearRegression[][]`	`initRegressions()` Helper function to initialize m_regressions.
`protected double`	`negativeLogLikelihood(double[][] dataYs, double[][] probs)` Returns the negative loglikelihood of the Y-values (actual class probabilities) given the p-values (current probability estimates).
`double`	`percentAttributesUsed()` Returns the fraction of all attributes in the data that are used in the logistic model (in percent).
`protected void`	`performBoosting()` Runs LogitBoost using the stopping criterion on the training set.
`protected int`	`performBoosting(Instances train, Instances test, double[] error, int maxIterations)` Runs LogitBoost on a training set and monitors the error on a test set.
`protected void`	`performBoosting(int numIterations)` Runs LogitBoost with a fixed number of iterations.
`protected void`	`performBoostingCV()` Runs LogitBoost, determining the best number of iterations by cross-validation.
`protected void`	`performBoostingInfCriterion()` Runs LogitBoost, determining the best number of iterations by an information criterion (currently AIC).
`protected boolean`	`performIteration(int iteration, double[][] trainYs, double[][] trainFs, double[][] probs, Instances trainNumeric)` Performs a single iteration of LogitBoost, and updates the model accordingly.
`protected double[]`	`probs(double[] Fs)` Computes the p-values (probabilities for the classes) from the F-values of the logistic model.
`void`	`setHeuristicStop(int heuristicStop)` Sets the option "heuristicStop".
`void`	`setMaxIterations(int maxIterations)` Sets the parameter "maxIterations".
`void`	`setUseAIC(boolean c)` Set the value of useAIC.
`void`	`setWeightTrimBeta(double w)` Sets the option "weightTrimBeta".
`java.lang.String`	`toString()` Returns a description of the logistic model (i.e., attributes and coefficients).

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

- Field Detail
  - m_numericDataHeader
```
protected Instances m_numericDataHeader
```
    Header-only version of the numeric version of the training data
  - m_numericData
```
protected Instances m_numericData
```
    Numeric version of the training data. Original class is replaced by a numeric pseudo-class.
  - m_train
```
protected Instances m_train
```
    Training data
  - m_useCrossValidation
```
protected boolean m_useCrossValidation
```
    Use cross-validation to determine best number of LogitBoost iterations ?
  - m_errorOnProbabilities
```
protected boolean m_errorOnProbabilities
```
    Use error on probabilities for stopping criterion of LogitBoost?
  - m_fixedNumIterations
```
protected int m_fixedNumIterations
```
    Use fixed number of iterations for LogitBoost? (if negative, cross-validate number of iterations)
  - m_heuristicStop
```
protected int m_heuristicStop
```
    Use heuristic to stop performing LogitBoost iterations earlier? If enabled, LogitBoost is stopped if the current (local) minimum of the error on a test set as a function of the number of iterations has not changed for m_heuristicStop iterations.
  - m_numRegressions
```
protected int m_numRegressions
```
    The number of LogitBoost iterations performed.
  - m_maxIterations
```
protected int m_maxIterations
```
    The maximum number of LogitBoost iterations
  - m_numClasses
```
protected int m_numClasses
```
    The number of different classes
  - m_regressions
```
protected SimpleLinearRegression[][] m_regressions
```
    Array holding the simple regression functions fit by LogitBoost
  - m_numFoldsBoosting
```
protected static int m_numFoldsBoosting
```
    Number of folds for cross-validating number of LogitBoost iterations
  - Z_MAX
```
protected static final double Z_MAX
```
    Threshold on the Z-value for LogitBoost
    
    See Also:
    
    Constant Field Values
  - m_numParameters
```
protected double m_numParameters
```
    Effective number of parameters used for AIC / BIC automatic stopping
  - m_weightTrimBeta
```
protected double m_weightTrimBeta
```
    Threshold for trimming weights. Instances with a weight lower than this (as a percentage of total weights) are not included in the regression fit.
- Constructor Detail
  - LogisticBase
```
public LogisticBase()
```
    Constructor that creates LogisticBase object with standard options.
  - LogisticBase
```
public LogisticBase(int numBoostingIterations,
                    boolean useCrossValidation,
                    boolean errorOnProbabilities)
```
    Constructor to create LogisticBase object.
    
    Parameters:
    
    numBoostingIterations - fixed number of iterations for LogitBoost (if negative, use cross-validation or stopping criterion on the training data).
    
    useCrossValidation - cross-validate number of LogitBoost iterations (if false, use stopping criterion on the training data).
    
    errorOnProbabilities - if true, use error on probabilities instead of misclassification for stopping criterion of LogitBoost
- Method Detail
  - buildClassifier
```
public void buildClassifier(Instances data)
                     throws java.lang.Exception
```
    Builds the logistic regression model usiing LogitBoost.
    
    Specified by:
    
    buildClassifier in interface Classifier
    
    Parameters:
    
    data - the training data
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - performBoostingCV
```
protected void performBoostingCV()
                          throws java.lang.Exception
```
    Runs LogitBoost, determining the best number of iterations by cross-validation.
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - copyRegressions
```
protected SimpleLinearRegression[][] copyRegressions(SimpleLinearRegression[][] a)
                                              throws java.lang.Exception
```
    Deep copies the given array of simple linear regression functions.
    
    Parameters:
    
    a - the array to copy
    
    Returns:
    
    the new array
    
    Throws:
    
    java.lang.Exception
  - performBoostingInfCriterion
```
protected void performBoostingInfCriterion()
                                    throws java.lang.Exception
```
    Runs LogitBoost, determining the best number of iterations by an information criterion (currently AIC).
    
    Throws:
    
    java.lang.Exception
  - performBoosting
```
protected int performBoosting(Instances train,
                              Instances test,
                              double[] error,
                              int maxIterations)
                       throws java.lang.Exception
```
    Runs LogitBoost on a training set and monitors the error on a test set. Used for running one fold when cross-validating the number of LogitBoost iterations.
    
    Parameters:
    
    train - the training set
    
    test - the test set
    
    error - array to hold the logged error values
    
    maxIterations - the maximum number of LogitBoost iterations to run
    
    Returns:
    
    the number of completed LogitBoost iterations (can be smaller than maxIterations if the heuristic for early stopping is active or there is a problem while fitting the regressions in LogitBoost).
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - performBoosting
```
protected void performBoosting(int numIterations)
                        throws java.lang.Exception
```
    Runs LogitBoost with a fixed number of iterations.
    
    Parameters:
    
    numIterations - the number of iterations to run
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - performBoosting
```
protected void performBoosting()
                        throws java.lang.Exception
```
    Runs LogitBoost using the stopping criterion on the training set. The number of iterations is used that gives the lowest error on the training set, either misclassification or error on probabilities (depending on the errorOnProbabilities option).
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getErrorRate
```
protected double getErrorRate(Instances data)
                       throws java.lang.Exception
```
    Returns the misclassification error of the current model on a set of instances.
    
    Parameters:
    
    data - the set of instances
    
    Returns:
    
    the error rate
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getMeanAbsoluteError
```
protected double getMeanAbsoluteError(Instances data)
                               throws java.lang.Exception
```
    Returns the error of the probability estimates for the current model on a set of instances.
    
    Parameters:
    
    data - the set of instances
    
    Returns:
    
    the error
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getBestIteration
```
protected int getBestIteration(double[] errors,
                               int maxIteration)
```
    Helper function to find the minimum in an array of error values.
    
    Parameters:
    
    errors - an array containing errors
    
    maxIteration - the maximum of iterations
    
    Returns:
    
    the minimum
  - performIteration
```
protected boolean performIteration(int iteration,
                                   double[][] trainYs,
                                   double[][] trainFs,
                                   double[][] probs,
                                   Instances trainNumeric)
                            throws java.lang.Exception
```
    Performs a single iteration of LogitBoost, and updates the model accordingly. A simple regression function is fit to the response and added to the m_regressions array.
    
    Parameters:
    
    iteration - the current iteration
    
    trainYs - the y-values (see description of LogitBoost) for the model trained so far
    
    trainFs - the F-values (see description of LogitBoost) for the model trained so far
    
    probs - the p-values (see description of LogitBoost) for the model trained so far
    
    trainNumeric - numeric version of the training data
    
    Returns:
    
    returns true if iteration performed successfully, false if no simple regression function could be fitted.
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - initRegressions
```
protected SimpleLinearRegression[][] initRegressions()
                                              throws java.lang.Exception
```
    Helper function to initialize m_regressions.
    
    Returns:
    
    the generated classifiers
    
    Throws:
    
    java.lang.Exception
  - getNumericData
```
protected Instances getNumericData(Instances data)
                            throws java.lang.Exception
```
    Converts training data to numeric version. The class variable is replaced by a pseudo-class used by LogitBoost.
    
    Parameters:
    
    data - the data to convert
    
    Returns:
    
    the converted data
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getZ
```
protected double getZ(double actual,
                      double p)
```
    Computes the LogitBoost response variable from y/p values (actual/estimated class probabilities).
    
    Parameters:
    
    actual - the actual class probability
    
    p - the estimated class probability
    
    Returns:
    
    the LogitBoost response
  - getZs
```
protected double[][] getZs(double[][] probs,
                           double[][] dataYs)
```
    Computes the LogitBoost response for an array of y/p values (actual/estimated class probabilities).
    
    Parameters:
    
    dataYs - the actual class probabilities
    
    probs - the estimated class probabilities
    
    Returns:
    
    the LogitBoost response
  - getWs
```
protected double[][] getWs(double[][] probs,
                           double[][] dataYs)
```
    Computes the LogitBoost weights from an array of y/p values (actual/estimated class probabilities).
    
    Parameters:
    
    dataYs - the actual class probabilities
    
    probs - the estimated class probabilities
    
    Returns:
    
    the LogitBoost weights
  - probs
```
protected double[] probs(double[] Fs)
```
    Computes the p-values (probabilities for the classes) from the F-values of the logistic model.
    
    Parameters:
    
    Fs - the F-values
    
    Returns:
    
    the p-values
  - getYs
```
protected double[][] getYs(Instances data)
```
    Computes the Y-values (actual class probabilities) for a set of instances.
    
    Parameters:
    
    data - the data to compute the Y-values from
    
    Returns:
    
    the Y-values
  - getFs
```
protected double[] getFs(Instance instance)
                  throws java.lang.Exception
```
    Computes the F-values for a single instance.
    
    Parameters:
    
    instance - the instance to compute the F-values for
    
    Returns:
    
    the F-values
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getFs
```
protected double[][] getFs(Instances data)
                    throws java.lang.Exception
```
    Computes the F-values for a set of instances.
    
    Parameters:
    
    data - the data to work on
    
    Returns:
    
    the F-values
    
    Throws:
    
    java.lang.Exception - if something goes wrong
  - getProbs
```
protected double[][] getProbs(double[][] dataFs)
```
    Computes the p-values (probabilities for the different classes) from the F-values for a set of instances.
    
    Parameters:
    
    dataFs - the F-values
    
    Returns:
    
    the p-values
  - negativeLogLikelihood
```
protected double negativeLogLikelihood(double[][] dataYs,
                                       double[][] probs)
```
    Returns the negative loglikelihood of the Y-values (actual class probabilities) given the p-values (current probability estimates).
    
    Parameters:
    
    dataYs - the Y-values
    
    probs - the p-values
    
    Returns:
    
    the likelihood
  - getUsedAttributes
```
public int[][] getUsedAttributes()
```
    Returns an array of the indices of the attributes used in the logistic model. The first dimension is the class, the second dimension holds a list of attribute indices. Attribute indices start at zero.
    
    Returns:
    
    the array of attribute indices
  - getNumRegressions
```
public int getNumRegressions()
```
    The number of LogitBoost iterations performed (= the number of simple regression functions fit).
    
    Returns:
    
    the number of LogitBoost iterations performed
  - getWeightTrimBeta
```
public double getWeightTrimBeta()
```
    Get the value of weightTrimBeta.
    
    Returns:
    
    Value of weightTrimBeta.
  - getUseAIC
```
public boolean getUseAIC()
```
    Get the value of useAIC.
    
    Returns:
    
    Value of useAIC.
  - setMaxIterations
```
public void setMaxIterations(int maxIterations)
```
    Sets the parameter "maxIterations".
    
    Parameters:
    
    maxIterations - the maximum iterations
  - setHeuristicStop
```
public void setHeuristicStop(int heuristicStop)
```
    Sets the option "heuristicStop".
    
    Parameters:
    
    heuristicStop - the heuristic stop to use
  - setWeightTrimBeta
```
public void setWeightTrimBeta(double w)
```
    Sets the option "weightTrimBeta".
  - setUseAIC
```
public void setUseAIC(boolean c)
```
    Set the value of useAIC.
    
    Parameters:
    
    c - Value to assign to useAIC.
  - getMaxIterations
```
public int getMaxIterations()
```
    Returns the maxIterations parameter.
    
    Returns:
    
    the maximum iteration
  - getCoefficients
```
protected double[][] getCoefficients()
```
    Returns an array holding the coefficients of the logistic model. First dimension is the class, the second one holds a list of coefficients. At position zero, the constant term of the model is stored, then, the coefficients for the attributes in ascending order.
    
    Returns:
    
    the array of coefficients
  - percentAttributesUsed
```
public double percentAttributesUsed()
```
    Returns the fraction of all attributes in the data that are used in the logistic model (in percent). An attribute is used in the model if it is used in any of the models for the different classes.
    
    Returns:
    
    the fraction of all attributes that are used
  - toString
```
public java.lang.String toString()
```
    Returns a description of the logistic model (i.e., attributes and coefficients).
    
    Overrides:
    
    toString in class java.lang.Object
    
    Returns:
    
    the description of the model
  - distributionForInstance
```
public double[] distributionForInstance(Instance instance)
                                 throws java.lang.Exception
```
    Returns class probabilities for an instance.
    
    Specified by:
    
    distributionForInstance in interface Classifier
    
    Overrides:
    
    distributionForInstance in class AbstractClassifier
    
    Parameters:
    
    instance - the instance to compute the distribution for
    
    Returns:
    
    the class probabilities
    
    Throws:
    
    java.lang.Exception - if distribution can't be computed successfully
  - cleanup
```
public void cleanup()
```
    Cleanup in order to save memory.
  - getRevision
```
public java.lang.String getRevision()
```
    Returns the revision string.
    
    Specified by:
    
    getRevision in interface RevisionHandler
    
    Overrides:
    
    getRevision in class AbstractClassifier
    
    Returns:
    
    the revision

Class LogisticBase

Field Summary

Fields inherited from class weka.classifiers.AbstractClassifier

Constructor Summary

Method Summary

Methods inherited from class weka.classifiers.AbstractClassifier

Methods inherited from class java.lang.Object

Field Detail

m_numericDataHeader

m_numericData

m_train

m_useCrossValidation

m_errorOnProbabilities

m_fixedNumIterations

m_heuristicStop

m_numRegressions

m_maxIterations

m_numClasses

m_regressions

m_numFoldsBoosting

Z_MAX

m_numParameters

m_weightTrimBeta

Constructor Detail

LogisticBase

LogisticBase

Method Detail

buildClassifier

performBoostingCV

copyRegressions

performBoostingInfCriterion

performBoosting

performBoosting

performBoosting

getErrorRate

getMeanAbsoluteError

getBestIteration

performIteration

initRegressions

getNumericData

getZ

getZs

getWs

probs

getYs

getFs

getFs

getProbs

negativeLogLikelihood

getUsedAttributes

getNumRegressions

getWeightTrimBeta

getUseAIC

setMaxIterations

setHeuristicStop

setWeightTrimBeta

setUseAIC

getMaxIterations

getCoefficients

percentAttributesUsed

toString

distributionForInstance

cleanup

getRevision