Package ai.libs.jaicore.ml.core.evaluation.evaluator

Class ExtrapolatedSaturationPointEvaluator

java.lang.Object

ai.libs.jaicore.ml.core.evaluation.evaluator.ExtrapolatedSaturationPointEvaluator

All Implemented Interfaces:

org.api4.java.ai.ml.core.evaluation.ISupervisedLearnerEvaluator<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>, org.api4.java.common.attributedobjects.IGetter<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,java.lang.Double>, org.api4.java.common.attributedobjects.IObjectEvaluator<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,java.lang.Double>

public class ExtrapolatedSaturationPointEvaluator
extends java.lang.Object
implements org.api4.java.ai.ml.core.evaluation.ISupervisedLearnerEvaluator<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>

For the classifier a learning curve will be extrapolated with a given set of anchorpoints. This learning curve can predict a saturation point with a tolerance epsilon. When a subsample is drawn at this saturation point it is the optimal trade-off between a fast training (therefore fast classifier evaluation) and dataset representability (therefore evaluation result expressiveness).

Constructor Summary

Constructors

Constructor

Description

ExtrapolatedSaturationPointEvaluator(int[] anchorpoints, ISamplingAlgorithmFactory<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>,? extends ASamplingAlgorithm<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>>> samplingAlgorithmFactory, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> train, double trainSplitForAnchorpointsMeasurement, LearningCurveExtrapolationMethod extrapolationMethod, long seed, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> test, org.api4.java.ai.ml.core.evaluation.supervised.loss.IDeterministicPredictionPerformanceMeasure<?,?> measure)

Create a classifier evaluator with an accuracy measurement at the extrapolated learning curves saturation point.

Method Summary

Modifier and Type	Method	Description
java.lang.Double	evaluate(org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>> learner)
void	setEpsilon(double epsilon)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.api4.java.common.attributedobjects.IObjectEvaluator

getPropertyOf

Constructor Detail

ExtrapolatedSaturationPointEvaluator

public ExtrapolatedSaturationPointEvaluator(int[] anchorpoints,
                                            ISamplingAlgorithmFactory<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>,? extends ASamplingAlgorithm<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>>> samplingAlgorithmFactory,
                                            org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> train,
                                            double trainSplitForAnchorpointsMeasurement,
                                            LearningCurveExtrapolationMethod extrapolationMethod,
                                            long seed,
                                            org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> test,
                                            org.api4.java.ai.ml.core.evaluation.supervised.loss.IDeterministicPredictionPerformanceMeasure<?,?> measure)

Create a classifier evaluator with an accuracy measurement at the extrapolated learning curves saturation point.

Parameters:

anchorpoints - Anchorpoints for the learning curve extrapolation.

samplingAlgorithmFactory - Subsampling factory for a subsampler to create samples at the given anchorpoints.

train - Dataset predict the learning curve with and where the subsample for the measurement is drawn from.

trainSplitForAnchorpointsMeasurement - Ratio to split the subsamples at the anchorpoints into train and test.

extrapolationMethod - Method to extrapolate a learning curve from the accuracy measurements at the anchorpoints.

seed - Random seed.

test - Test dataset to measure the accuracy.

Method Detail

setEpsilon

public void setEpsilon(double epsilon)

evaluate

public java.lang.Double evaluate(org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>> learner)
                          throws java.lang.InterruptedException,
                                 org.api4.java.common.attributedobjects.ObjectEvaluationFailedException

Specified by:

evaluate in interface org.api4.java.common.attributedobjects.IObjectEvaluator<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,java.lang.Double>

Throws:

java.lang.InterruptedException

org.api4.java.common.attributedobjects.ObjectEvaluationFailedException