Package ai.djl.training.optimizer

Class Optimizer

java.lang.Object

ai.djl.training.optimizer.Optimizer

Direct Known Subclasses:

Adadelta, Adagrad, Adam, AdamW, Nag, RmsProp, Sgd

public abstract class Optimizer
extends java.lang.Object

An Optimizer updates the weight parameters to minimize the loss function. Optimizer is an abstract class that provides the base implementation for optimizers.

See Also:

The D2L chapters on optimization algorithms

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	Optimizer.OptimizerBuilder<T extends Optimizer.OptimizerBuilder>	The Builder to construct an Optimizer.

Field Summary

Fields

Modifier and Type	Field	Description
protected float	clipGrad
protected float	rescaleGrad

Constructor Summary

Constructors

Constructor	Description
Optimizer(Optimizer.OptimizerBuilder<?> builder)	Creates a new instance of Optimizer.

Method Summary

Modifier and Type	Method	Description
static Adadelta.Builder	adadelta()	Returns a new instance of Adadelta.Builder that can build an Adadelta optimizer.
static Adagrad.Builder	adagrad()	Returns a new instance of Adagrad.Builder that can build an Adagrad optimizer.
static Adam.Builder	adam()	Returns a new instance of Adam.Builder that can build an Adam optimizer.
static AdamW.Builder	adamW()	Returns a new instance of AdamW.Builder that can build an AdamW optimizer.
protected float	getWeightDecay()	Gets the value of weight decay.
static Nag.Builder	nag()	Returns a new instance of Nag.Builder that can build an Nag optimizer.
static RmsProp.Builder	rmsprop()	Returns a new instance of RmsProp.Builder that can build an RmsProp optimizer.
static Sgd.Builder	sgd()	Returns a new instance of Sgd.Builder that can build an Sgd optimizer.
abstract void	update(java.lang.String parameterId, NDArray weight, NDArray grad)	Updates the parameters according to the gradients.
protected int	updateCount(java.lang.String parameterId)
protected NDArray	withDefaultState(java.util.Map<java.lang.String,java.util.Map<Device,NDArray>> state, java.lang.String key, Device device, java.util.function.Function<java.lang.String,NDArray> defaultFunction)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

rescaleGrad

protected float rescaleGrad

clipGrad

protected float clipGrad

Constructor Detail

Optimizer

public Optimizer(Optimizer.OptimizerBuilder<?> builder)

Creates a new instance of Optimizer.

Parameters:

builder - the builder used to create an instance of Optimizer

Method Detail

sgd

public static Sgd.Builder sgd()

Returns a new instance of Sgd.Builder that can build an Sgd optimizer.

Returns:

the Sgd Sgd.Builder

nag

public static Nag.Builder nag()

Returns a new instance of Nag.Builder that can build an Nag optimizer.

Returns:

the Nag Nag.Builder

adam

public static Adam.Builder adam()

Returns a new instance of Adam.Builder that can build an Adam optimizer.

Returns:

the Adam Adam.Builder

adamW

public static AdamW.Builder adamW()

Returns a new instance of AdamW.Builder that can build an AdamW optimizer.

Returns:

the AdamW AdamW.Builder

rmsprop

public static RmsProp.Builder rmsprop()

Returns a new instance of RmsProp.Builder that can build an RmsProp optimizer.

Returns:

the RmsProp RmsProp.Builder

adagrad

public static Adagrad.Builder adagrad()

Returns a new instance of Adagrad.Builder that can build an Adagrad optimizer.

Returns:

the Adagrad Adagrad.Builder

adadelta

public static Adadelta.Builder adadelta()

Returns a new instance of Adadelta.Builder that can build an Adadelta optimizer.

Returns:

the Adadelta Adadelta.Builder

getWeightDecay

protected float getWeightDecay()

Gets the value of weight decay.

Returns:

the value of weight decay

updateCount

protected int updateCount(java.lang.String parameterId)

update

public abstract void update(java.lang.String parameterId,
                            NDArray weight,
                            NDArray grad)

Updates the parameters according to the gradients.

Parameters:

parameterId - the parameter to be updated

weight - the weights of the parameter

grad - the gradients

withDefaultState

protected NDArray withDefaultState(java.util.Map<java.lang.String,java.util.Map<Device,NDArray>> state,
                                   java.lang.String key,
                                   Device device,
                                   java.util.function.Function<java.lang.String,NDArray> defaultFunction)