Package ai.djl.training.evaluator

Class Evaluator

  • Direct Known Subclasses:
    AbstractAccuracy, BoundingBoxError, IndexEvaluator, Loss
    public abstract class Evaluator
extends java.lang.Object
    Base class for all Evaluators that can be used to evaluate the performance of a model.

    The Evaluators can all be monitored to make an assessment about the performance of the model. However, only ones that further extend Loss are suited to being used to directly optimize a model.

    In addition to computing the evaluation, an evaluator can accumulate values to compute a total evaluation. For different purposes, it is possible to have multiple accumulators on a single evaluator. Each accumulator must be added with a String key to identify the accumulator. Before using an accumulator, you must addAccumulator(String). Then, call updateAccumulator(String, NDList, NDList) to add more data to the accumulator. You can use getAccumulator(String) to retrieve the accumulated value and resetAccumulator(String) to reset the accumulator to the same value as when just added.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected java.util.Map<java.lang.String,​java.lang.Long> totalInstances  

    • Constructor Summary

      Constructors 
      Constructor Description
      Evaluator​(java.lang.String name)
      Creates an evaluator with abstract update methods.

    • Method Summary

      All Methods Instance Methods Abstract Methods Concrete Methods 
      Modifier and Type Method Description
      abstract void addAccumulator​(java.lang.String key)
      Adds an accumulator for the results of the evaluation with the given key.
      protected void checkLabelShapes​(NDArray labels, NDArray predictions)
      Checks the length of NDArrays.
      protected void checkLabelShapes​(NDArray labels, NDArray predictions, boolean checkDimOnly)
      Checks if the two input NDArray have the same length or shape.
      abstract NDArray evaluate​(NDList labels, NDList predictions)
      Calculates the evaluation between the labels and the predictions.
      abstract float getAccumulator​(java.lang.String key)
      Returns the accumulated evaluator value.
      java.lang.String getName()
      Returns the name of this Evaluator.
      abstract void resetAccumulator​(java.lang.String key)
      Resets the evaluator value with the given key.
      abstract void updateAccumulator​(java.lang.String key, NDList labels, NDList predictions)
      Updates the evaluator with the given key based on a NDList of labels and predictions.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • totalInstances

        protected java.util.Map<java.lang.String,​java.lang.Long> totalInstances

    • Constructor Detail

      • Evaluator

        public Evaluator​(java.lang.String name)
        Creates an evaluator with abstract update methods.
        Parameters:
        name - the name of the evaluator

    • Method Detail

      • getName

        public java.lang.String getName()
        Returns the name of this Evaluator.
        Returns:
        the name of this Evaluator

      • evaluate

        public abstract NDArray evaluate​(NDList labels,
                                 NDList predictions)
        Calculates the evaluation between the labels and the predictions.
        Parameters:
        labels - the correct values
        predictions - the predicted values
        Returns:
        the evaluation result

      • addAccumulator

        public abstract void addAccumulator​(java.lang.String key)
        Adds an accumulator for the results of the evaluation with the given key.
        Parameters:
        key - the key for the new accumulator

      • updateAccumulator

        public abstract void updateAccumulator​(java.lang.String key,
                                       NDList labels,
                                       NDList predictions)
        Updates the evaluator with the given key based on a NDList of labels and predictions.

        This is a synchronized operation. You should only call it at the end of a batch or epoch.

        Parameters:
        key - the key of the accumulator to update
        labels - a NDList of labels
        predictions - a NDList of predictions

      • resetAccumulator

        public abstract void resetAccumulator​(java.lang.String key)
        Resets the evaluator value with the given key.
        Parameters:
        key - the key of the accumulator to reset

      • getAccumulator

        public abstract float getAccumulator​(java.lang.String key)
        Returns the accumulated evaluator value.
        Parameters:
        key - the key of the accumulator to get
        Returns:
        the accumulated value
        Throws:
        java.lang.IllegalArgumentException - if no accumulator was added with the given key

      • checkLabelShapes

        protected void checkLabelShapes​(NDArray labels,
                                NDArray predictions,
                                boolean checkDimOnly)
        Checks if the two input NDArray have the same length or shape.
        Parameters:
        labels - a NDArray of labels
        predictions - a NDArray of predictions
        checkDimOnly - whether to check for first dimension only

      • checkLabelShapes

        protected void checkLabelShapes​(NDArray labels,
                                NDArray predictions)
        Checks the length of NDArrays.
        Parameters:
        labels - a NDArray of labels
        predictions - a NDArray of predictions