ai.h2o.automl

Class ModelingStep<M extends hex.Model>

java.lang.Object

water.Iced<ModelingStep>

ai.h2o.automl.ModelingStep<M>

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable<ModelingStep>

Direct Known Subclasses:

ModelingStep.DynamicStep, ModelingStep.GridStep, ModelingStep.ModelStep, ModelingStep.SelectionStep

public abstract class ModelingStep<M extends hex.Model>
extends water.Iced<ModelingStep>

Parent class defining common properties and common logic for actual AutoML training steps.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ModelingStep.DynamicStep<M extends hex.Model> Step designed to dynamically choose to train a model or another, a grid or anything else, based on the current automl workflow history.
static class	ModelingStep.GridStep<M extends hex.Model> Step designed to build multiple models using a (random) grid search.
static class	ModelingStep.ModelStep<M extends hex.Model> Step designed to build a single/default model.
protected static class	ModelingStep.SeedPolicy
static class	ModelingStep.SelectionStep<M extends hex.Model> Step designed to train some models (or not) and then deciding to make a selection and add and/or remove models to/from the current leaderboard.

Field Summary

Fields

Modifier and Type	Field and Description
protected IAlgo	_algo
protected java.lang.String	_description
protected java.lang.String	_id
protected AutoML.Constraint[]	_ignoredConstraints
protected int	_priorityGroup
protected java.lang.String	_provider
protected int	_weight
protected WorkAllocations.Work	_work

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	ModelingStep(java.lang.String provider, IAlgo algo, java.lang.String id, int priorityGroup, int weight, AutoML autoML)

Method Summary

Modifier and Type	Method and Description
protected AutoML	aml()
protected void	applyPreprocessing(hex.Model.Parameters params)
boolean	canRun()
boolean	equals(java.lang.Object o)
IAlgo	getAlgo()
protected WorkAllocations.Work	getAllocatedWork()
java.lang.String	getGlobalId()
java.lang.String	getId()
protected abstract WorkAllocations.JobType	getJobType()
protected PreprocessingConfig	getPreprocessingConfig()
int	getPriorityGroup()
java.lang.String	getProvider() Each provider (usually one class) defining a collection of steps must have a unique name.
protected java.util.Optional<? extends ModelingStep>	getSubStep(java.lang.String id)
protected hex.Model[]	getTrainedModels()
protected water.Key<hex.Model>[]	getTrainedModelsKeys()
int	getWeight()
protected WorkAllocations	getWorkAllocations()
int	hashCode()
boolean	ignores(AutoML.Constraint constraint)
protected void	initTimeConstraints(hex.Model.Parameters parms, double upperLimit)
protected boolean	isCVEnabled()
boolean	isResumable()
java.util.Iterator<? extends ModelingStep>	iterateSubSteps()
boolean	limitModelTrainingTime()
protected water.Key	makeKey(java.lang.String name, boolean withCounter)
protected WorkAllocations.Work	makeWork() Creates the WorkAllocations.Work instance representing the total work handled by this step.
protected void	onDone(water.Job job)
protected void	register(water.Key key)
water.Job	run() Execute this modeling step, returning the job associated to it if any.
protected void	setClassBalancingParams(hex.Model.Parameters params)
protected void	setCommonModelBuilderParams(hex.Model.Parameters params) Assign common parameters to the model params before building the model or set of models.
protected void	setCrossValidationParams(hex.Model.Parameters params)
protected void	setCustomParams(hex.Model.Parameters params)
protected void	setDistributionParameters(hex.Model.Parameters parms)
protected void	setSeed(hex.Model.Parameters parms, hex.Model.Parameters defaults, ModelingStep.SeedPolicy seedPolicy)
protected void	setStoppingCriteria(hex.Model.Parameters parms, hex.Model.Parameters defaults) Configures early-stopping for the model or set of models to be built.
protected void	setWeightingParams(hex.Model.Parameters params)
protected abstract water.Job	startJob() Starts a new Job as part of this step.
protected <MP extends hex.Model.Parameters> water.Job<M>	startModel(water.Key<M> resultKey, MP params)
protected <MP extends hex.Model.Parameters> water.Job<hex.grid.Grid>	startSearch(water.Key<hex.grid.Grid> resultKey, MP baseParams, java.util.Map<java.lang.String,java.lang.Object[]> hyperParams, hex.grid.HyperSpaceSearchCriteria searchCriteria)

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Field Detail

_algo

protected final IAlgo _algo

_provider

protected final java.lang.String _provider

_id

protected final java.lang.String _id

_weight

protected int _weight

_priorityGroup

protected int _priorityGroup

_ignoredConstraints

protected AutoML.Constraint[] _ignoredConstraints

_description

protected java.lang.String _description

_work

protected WorkAllocations.Work _work

Constructor Detail

ModelingStep

protected ModelingStep(java.lang.String provider,
                       IAlgo algo,
                       java.lang.String id,
                       int priorityGroup,
                       int weight,
                       AutoML autoML)

Method Detail

startSearch

protected <MP extends hex.Model.Parameters> water.Job<hex.grid.Grid> startSearch(water.Key<hex.grid.Grid> resultKey,
                                                                                 MP baseParams,
                                                                                 java.util.Map<java.lang.String,java.lang.Object[]> hyperParams,
                                                                                 hex.grid.HyperSpaceSearchCriteria searchCriteria)

startModel

protected <MP extends hex.Model.Parameters> water.Job<M> startModel(water.Key<M> resultKey,
                                                                    MP params)

setDistributionParameters

protected void setDistributionParameters(hex.Model.Parameters parms)

getProvider

public java.lang.String getProvider()

Each provider (usually one class) defining a collection of steps must have a unique name.

Returns:

the name of the provider (usually simply the name of an algo) defining this step.

getId

public java.lang.String getId()

Returns:

the step identifier: should be unique inside its provider.

getGlobalId

public java.lang.String getGlobalId()

Returns:

a string that identifies the step uniquely among all steps defined by all providers.

getAlgo

public IAlgo getAlgo()

getWeight

public int getWeight()

getPriorityGroup

public int getPriorityGroup()

isResumable

public boolean isResumable()

ignores

public boolean ignores(AutoML.Constraint constraint)

limitModelTrainingTime

public boolean limitModelTrainingTime()

canRun

public boolean canRun()

Returns:

true iff we can call run() on this modeling step to start a new job.

run

public water.Job run()

Execute this modeling step, returning the job associated to it if any.

Returns:

iterateSubSteps

public java.util.Iterator<? extends ModelingStep> iterateSubSteps()

Returns:

an Iterator for the potential sub-steps provided by this modeling step.

getSubStep

protected java.util.Optional<? extends ModelingStep> getSubStep(java.lang.String id)

Parameters:

id -

Returns:

the sub-step (if any) with the given identifier, or null if there's no sub-step

getJobType

protected abstract WorkAllocations.JobType getJobType()

startJob

protected abstract water.Job startJob()

Starts a new Job as part of this step.

Returns:

the newly started job.

onDone

protected void onDone(water.Job job)

register

protected void register(water.Key key)

aml

protected AutoML aml()

getAllocatedWork

protected WorkAllocations.Work getAllocatedWork()

Returns:

the total work allocated for this step.

makeWork

protected WorkAllocations.Work makeWork()

Creates the WorkAllocations.Work instance representing the total work handled by this step.

Returns:

makeKey

protected water.Key makeKey(java.lang.String name,
                            boolean withCounter)

getWorkAllocations

protected WorkAllocations getWorkAllocations()

getTrainedModels

protected hex.Model[] getTrainedModels()

Returns:

the models trained until now, sorted by the default leaderboard metric.

getTrainedModelsKeys

protected water.Key<hex.Model>[] getTrainedModelsKeys()

isCVEnabled

protected boolean isCVEnabled()

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class java.lang.Object

hashCode

public int hashCode()

Overrides:

hashCode in class java.lang.Object

setCommonModelBuilderParams

protected void setCommonModelBuilderParams(hex.Model.Parameters params)

Assign common parameters to the model params before building the model or set of models. This includes:

• data-related parameters: frame/columns parameters, class distribution.
• cross-validation parameters/
• memory-optimization: if certain objects build during training should be kept after training or not/
• model management: checkpoints.

Parameters:

params - the model parameters to which the common parameters will be added.

setCrossValidationParams

protected void setCrossValidationParams(hex.Model.Parameters params)

setWeightingParams

protected void setWeightingParams(hex.Model.Parameters params)

setClassBalancingParams

protected void setClassBalancingParams(hex.Model.Parameters params)

setCustomParams

protected void setCustomParams(hex.Model.Parameters params)

applyPreprocessing

protected void applyPreprocessing(hex.Model.Parameters params)

getPreprocessingConfig

protected PreprocessingConfig getPreprocessingConfig()

setStoppingCriteria

protected void setStoppingCriteria(hex.Model.Parameters parms,
                                   hex.Model.Parameters defaults)

Configures early-stopping for the model or set of models to be built.

Parameters:

parms - the model parameters to which the stopping criteria will be added.

defaults - the default parameters for the corresponding ModelBuilder.

setSeed

protected void setSeed(hex.Model.Parameters parms,
                       hex.Model.Parameters defaults,
                       ModelingStep.SeedPolicy seedPolicy)

Parameters:

parms - the model parameters to which the stopping criteria will be added.

defaults - the default parameters for the corresponding ModelBuilder.

seedPolicy - the policy defining how the seed will be assigned to the model parameters.

initTimeConstraints

protected void initTimeConstraints(hex.Model.Parameters parms,
                                   double upperLimit)