Package ai.libs.jaicore.ml.core.evaluation.evaluator

Class LearningCurveExtrapolationEvaluator

  • All Implemented Interfaces:
    org.api4.java.ai.ml.classification.IClassifierEvaluator, org.api4.java.ai.ml.core.evaluation.ISupervisedLearnerEvaluator<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>, org.api4.java.common.attributedobjects.IGetter<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,​java.lang.Double>, org.api4.java.common.attributedobjects.IObjectEvaluator<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,​java.lang.Double>, org.api4.java.common.control.ILoggingCustomizable, org.api4.java.common.event.IEventEmitter<java.lang.Object>
    public class LearningCurveExtrapolationEvaluator
extends java.lang.Object
implements org.api4.java.ai.ml.classification.IClassifierEvaluator, org.api4.java.common.control.ILoggingCustomizable, org.api4.java.common.event.IEventEmitter<java.lang.Object>
    Evaluates a classifier by predicting its learning curve with a few anchorpoints. The evaluation result is the accuracy or the error rate (configurable) for the complete dataset. Depending on the chosen anchorpoints this evaluation method will be really fast, but can be inaccurate depending on the learning curve extrapolation method, since it will only give a prediction of the accuracy and does not measure it.

    • Constructor Summary

      Constructors 
      Constructor Description
      LearningCurveExtrapolationEvaluator​(int[] anchorpoints, ISamplingAlgorithmFactory<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>,​? extends ASamplingAlgorithm<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>>> samplingAlgorithmFactory, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dataset, double trainSplitForAnchorpointsMeasurement, LearningCurveExtrapolationMethod extrapolationMethod, long seed)
      Create a classifier evaluator with learning curve extrapolation.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      java.lang.Double evaluate​(org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>> classifier)
      Computes the (estimated) measure of the classifier on the full dataset
      java.lang.String getLoggerName()  
      void registerListener​(java.lang.Object listener)
      Register observers for learning curve predictions (including estimates of the time)
      void setFullDatasetSize​(int fullDatasetSize)  
      void setLoggerName​(java.lang.String name)  

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface org.api4.java.common.attributedobjects.IObjectEvaluator

        getPropertyOf

    • Constructor Detail

      • LearningCurveExtrapolationEvaluator

        public LearningCurveExtrapolationEvaluator​(int[] anchorpoints,
                                           ISamplingAlgorithmFactory<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>,​? extends ASamplingAlgorithm<org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?>>> samplingAlgorithmFactory,
                                           org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dataset,
                                           double trainSplitForAnchorpointsMeasurement,
                                           LearningCurveExtrapolationMethod extrapolationMethod,
                                           long seed)
        Create a classifier evaluator with learning curve extrapolation.
        Parameters:
        anchorpoints - Anchorpoints for the learning curve extrapolation.
        samplingAlgorithmFactory - Subsampling factory to create a subsampler for the samples at the given anchorpoints.
        dataset - Dataset to evaluate the classifier with.
        trainSplitForAnchorpointsMeasurement - Ratio to split the subsamples at the anchorpoints into train and test.
        extrapolationMethod - Method to extrapolate a learning curve from the accuracy measurements at the anchorpoints.
        seed - Random seed.

    • Method Detail

      • setFullDatasetSize

        public void setFullDatasetSize​(int fullDatasetSize)

      • evaluate

        public java.lang.Double evaluate​(org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>> classifier)
                          throws java.lang.InterruptedException,
                                 org.api4.java.common.attributedobjects.ObjectEvaluationFailedException
        Computes the (estimated) measure of the classifier on the full dataset
        Specified by:
        evaluate in interface org.api4.java.common.attributedobjects.IObjectEvaluator<org.api4.java.ai.ml.core.learner.ISupervisedLearner<org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance,​org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<? extends org.api4.java.ai.ml.core.dataset.supervised.ILabeledInstance>>,​java.lang.Double>
        Throws:
        java.lang.InterruptedException
        org.api4.java.common.attributedobjects.ObjectEvaluationFailedException

      • getLoggerName

        public java.lang.String getLoggerName()
        Specified by:
        getLoggerName in interface org.api4.java.common.control.ILoggingCustomizable

      • setLoggerName

        public void setLoggerName​(java.lang.String name)
        Specified by:
        setLoggerName in interface org.api4.java.common.control.ILoggingCustomizable

      • registerListener

        public void registerListener​(java.lang.Object listener)
        Register observers for learning curve predictions (including estimates of the time)
        Specified by:
        registerListener in interface org.api4.java.common.event.IEventEmitter<java.lang.Object>
        Parameters:
        listener -