Package ai.libs.mlplan.safeguard

Interface IEvaluationSafeGuard

All Known Implementing Classes:

AlwaysEvaluateSafeGuard, AlwaysPreventSafeGuard

public interface IEvaluationSafeGuard

An evaluation safe guard can be used to predict whether an evaluation is likely to succeed or not. Based on this information the evaluation may be adapted or omitted totally. Three types of runtimes are distinguished: - induction time: The time needed to induce a model as described via the component instance. - inference time: The time needed to do inference with a model as described via the component instance. - evaluation time: The time needed to do both: induction and inference. Thus, it is the sum of the previous two runtimes.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	ANNOTATION_PREDICTED_INDUCTION_TIME
static java.lang.String	ANNOTATION_PREDICTED_INFERENCE_TIME
static java.lang.String	ANNOTATION_SOURCE

Method Summary

Modifier and Type	Method	Description
default double	predictEvaluationTime(ai.libs.jaicore.components.model.ComponentInstance ci, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTest)
double	predictInductionTime(ai.libs.jaicore.components.model.ComponentInstance ci, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain)	Predicts the runtime that is required for inducing a model.
double	predictInferenceTime(ai.libs.jaicore.components.model.ComponentInstance ci, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain, org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTest)	Predicts the runtime that is required for doing inference with the given model.
boolean	predictWillAdhereToTimeout(ai.libs.jaicore.components.model.ComponentInstance ci, org.api4.java.algorithm.Timeout timeout)	Predicts whether a component instance ci is likely to adhere to the given timeout.
void	registerListener(java.lang.Object listener)
void	updateWithActualInformation(ai.libs.jaicore.components.model.ComponentInstance ci, ITimeTrackingLearner wrappedLearner)	Updates the safe guard with current information obtained by measuring the induction and inference time of the given component instance on-line.

Field Detail

ANNOTATION_PREDICTED_INDUCTION_TIME

static final java.lang.String ANNOTATION_PREDICTED_INDUCTION_TIME

See Also:

Constant Field Values

ANNOTATION_PREDICTED_INFERENCE_TIME

static final java.lang.String ANNOTATION_PREDICTED_INFERENCE_TIME

See Also:

Constant Field Values

ANNOTATION_SOURCE

static final java.lang.String ANNOTATION_SOURCE

See Also:

Constant Field Values

Method Detail

predictWillAdhereToTimeout

boolean predictWillAdhereToTimeout(ai.libs.jaicore.components.model.ComponentInstance ci,
                                   org.api4.java.algorithm.Timeout timeout)
                            throws java.lang.Exception

Predicts whether a component instance ci is likely to adhere to the given timeout.

Parameters:

timeout - The timeout posed to the evaluation of the component instance.

Returns:

Returns true iff the component instance can likely be evaluated within the given timeout.

Throws:

java.lang.Exception

predictInductionTime

double predictInductionTime(ai.libs.jaicore.components.model.ComponentInstance ci,
                            org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain)
                     throws java.lang.Exception

Predicts the runtime that is required for inducing a model.

Parameters:

ci - The component instance describing the model to predict the induction time for.

metaFeaturesTrain - The meta features describing the data inducing a model from.

Returns:

The time needed for inducing a model.

Throws:

java.lang.Exception

predictInferenceTime

double predictInferenceTime(ai.libs.jaicore.components.model.ComponentInstance ci,
                            org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain,
                            org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTest)
                     throws java.lang.Exception

Predicts the runtime that is required for doing inference with the given model.

Parameters:

ci - The component instance describing the model to predict the inference time for.

metaFeaturesTest - The meta features describing the data for which inference is to be done.

Returns:

The time needed for making predictions on the validation set.

Throws:

java.lang.Exception

predictEvaluationTime

default double predictEvaluationTime(ai.libs.jaicore.components.model.ComponentInstance ci,
                                     org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTrain,
                                     org.api4.java.ai.ml.core.dataset.supervised.ILabeledDataset<?> dTest)
                              throws java.lang.Exception

Parameters:

ci - The component instance describing the model to predict the evaluation time for.

metaFeaturesTrain - The meta features describing the data to induce a model from.

metaFeaturesTest - The meta features describing the data to do inference for.

Returns:

The time needed for inducing a model and making predictions.

Throws:

java.lang.Exception

updateWithActualInformation

void updateWithActualInformation(ai.libs.jaicore.components.model.ComponentInstance ci,
                                 ITimeTrackingLearner wrappedLearner)

Updates the safe guard with current information obtained by measuring the induction and inference time of the given component instance on-line.

Parameters:

ci - The component instance describing the model to update the actual information for.

wrappedLearner - The learner that has been used to evaluate the component instance. It must be a time tracking learner.

registerListener

void registerListener(java.lang.Object listener)