ai.libs.jaicore.ml.evaluation.evaluators.weka

Class LearningCurveExtrapolationEvaluator

java.lang.Object

ai.libs.jaicore.ml.evaluation.evaluators.weka.LearningCurveExtrapolationEvaluator

All Implemented Interfaces:

ai.libs.jaicore.basic.events.IEventEmitter, ai.libs.jaicore.basic.ILoggingCustomizable, ai.libs.jaicore.basic.IObjectEvaluator<weka.classifiers.Classifier,java.lang.Double>, IClassifierEvaluator

public class LearningCurveExtrapolationEvaluator
extends java.lang.Object
implements IClassifierEvaluator, ai.libs.jaicore.basic.ILoggingCustomizable, ai.libs.jaicore.basic.events.IEventEmitter

Evaluates a classifier by predicting its learning curve with a few anchorpoints. The evaluation result is the accuracy or the error rate (configurable) for the complete dataset. Depending on the chosen anchorpoints this evaluation method will be really fast, but can be inaccurate depending on the learning curve extrapolation method, since it will only give a prediction of the accuracy and does not measure it.

Constructor Summary

Constructors

Constructor and Description
LearningCurveExtrapolationEvaluator(int[] anchorpoints, ISamplingAlgorithmFactory<IInstance,? extends ASamplingAlgorithm<IInstance>> samplingAlgorithmFactory, IDataset<? extends IInstance> dataset, double trainSplitForAnchorpointsMeasurement, LearningCurveExtrapolationMethod extrapolationMethod, long seed) Create a classifier evaluator with learning curve extrapolation.

Method Summary

Modifier and Type	Method and Description
java.lang.Double	evaluate(weka.classifiers.Classifier classifier) Computes the (estimated) measure of the classifier on the full dataset
java.lang.String	getLoggerName()
void	registerListener(java.lang.Object listener) Register observers for learning curve predictions (including estimates of the time)
void	setFullDatasetSize(int fullDatasetSize)
void	setLoggerName(java.lang.String name)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

LearningCurveExtrapolationEvaluator

public LearningCurveExtrapolationEvaluator(int[] anchorpoints,
                                           ISamplingAlgorithmFactory<IInstance,? extends ASamplingAlgorithm<IInstance>> samplingAlgorithmFactory,
                                           IDataset<? extends IInstance> dataset,
                                           double trainSplitForAnchorpointsMeasurement,
                                           LearningCurveExtrapolationMethod extrapolationMethod,
                                           long seed)

Create a classifier evaluator with learning curve extrapolation.

Parameters:

anchorpoints - Anchorpoints for the learning curve extrapolation.

samplingAlgorithmFactory - Subsampling factory to create a subsampler for the samples at the given anchorpoints.

dataset - Dataset to evaluate the classifier with.

trainSplitForAnchorpointsMeasurement - Ratio to split the subsamples at the anchorpoints into train and test.

extrapolationMethod - Method to extrapolate a learning curve from the accuracy measurements at the anchorpoints.

seed - Random seed.

Method Detail

setFullDatasetSize

public void setFullDatasetSize(int fullDatasetSize)

evaluate

public java.lang.Double evaluate(weka.classifiers.Classifier classifier)
                          throws java.lang.InterruptedException,
                                 ai.libs.jaicore.basic.algorithm.exceptions.ObjectEvaluationFailedException

Computes the (estimated) measure of the classifier on the full dataset

Specified by:

evaluate in interface ai.libs.jaicore.basic.IObjectEvaluator<weka.classifiers.Classifier,java.lang.Double>

Throws:

java.lang.InterruptedException

ai.libs.jaicore.basic.algorithm.exceptions.ObjectEvaluationFailedException

getLoggerName

public java.lang.String getLoggerName()

Specified by:

getLoggerName in interface ai.libs.jaicore.basic.ILoggingCustomizable

setLoggerName

public void setLoggerName(java.lang.String name)

Specified by:

setLoggerName in interface ai.libs.jaicore.basic.ILoggingCustomizable

registerListener

public void registerListener(java.lang.Object listener)

Register observers for learning curve predictions (including estimates of the time)

Specified by:

registerListener in interface ai.libs.jaicore.basic.events.IEventEmitter

Parameters:

listener -