ai.djl.nn.recurrent

Class LSTM

java.lang.Object

ai.djl.nn.AbstractBlock

ai.djl.nn.ParameterBlock

ai.djl.nn.recurrent.RecurrentCell

ai.djl.nn.recurrent.LSTM

All Implemented Interfaces:

Block

public class LSTM
extends RecurrentCell

Applies Long Short-Term Memory recurrent layer to input.

Reference paper - LONG SHORT-TERM MEMORY - Hochreiter, 1997. http://www.bioinf.jku.at/publications/older/2604.pdf

$$ \begin{split}\begin{array}{ll} i_t = \mathrm{sigmoid}(W_{ii} x_t + b_{ii} + W_{hi} h_{(t-1)} + b_{hi}) \\ f_t = \mathrm{sigmoid}(W_{if} x_t + b_{if} + W_{hf} h_{(t-1)} + b_{hf}) \\ g_t = \tanh(W_{ig} x_t + b_{ig} + W_{hc} h_{(t-1)} + b_{hg}) \\ o_t = \mathrm{sigmoid}(W_{io} x_t + b_{io} + W_{ho} h_{(t-1)} + b_{ho}) \\ c_t = f_t * c_{(t-1)} + i_t * g_t \\ h_t = o_t * \tanh(c_t) \end{array}\end{split} $$

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LSTM.Builder The Builder to construct a LSTM type of Block.

Nested classes/interfaces inherited from class ai.djl.nn.recurrent.RecurrentCell

RecurrentCell.BaseBuilder<T extends RecurrentCell.BaseBuilder>

Field Summary

Fields inherited from class ai.djl.nn.recurrent.RecurrentCell

currentVersion, dropRate, gates, mode, numStackedLayers, parameters, stateOutputs, stateShape, stateSize, useBidirectional, useSequenceLength

Fields inherited from class ai.djl.nn.AbstractBlock

inputNames, inputShapes

Method Summary

Modifier and Type	Method and Description
static LSTM.Builder	builder() Creates a builder to build a LSTM.
NDList	forward(ParameterStore parameterStore, NDList inputs, ai.djl.util.PairList<java.lang.String,java.lang.Object> params) Applies the operating function of the block once.
void	loadParameters(NDManager manager, java.io.DataInputStream is) Loads the parameters from the given input stream.
protected NDList	opInputs(ParameterStore parameterStore, NDList inputs)
void	saveParameters(java.io.DataOutputStream os) Writes the parameters of the block to the given outputStream.

Methods inherited from class ai.djl.nn.recurrent.RecurrentCell

beforeInitialize, getDirectParameters, getOutputShapes, getParameterShape, updateInputLayoutToTNC, validateInputSize

Methods inherited from class ai.djl.nn.ParameterBlock

getChildren, initialize, toString

Methods inherited from class ai.djl.nn.AbstractBlock

cast, clear, describeInput, getParameters, isInitialized, setInitializer, setInitializer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ai.djl.nn.Block

forward, validateLayout

Method Detail

forward

public NDList forward(ParameterStore parameterStore,
                      NDList inputs,
                      ai.djl.util.PairList<java.lang.String,java.lang.Object> params)

Applies the operating function of the block once. This method should be called only on blocks that are initialized.

Specified by:

forward in interface Block

Overrides:

forward in class RecurrentCell

Parameters:

parameterStore - the parameter store

inputs - the input NDList

params - optional parameters

Returns:

the output of the forward pass

opInputs

protected NDList opInputs(ParameterStore parameterStore,
                          NDList inputs)

Overrides:

opInputs in class RecurrentCell

saveParameters

public void saveParameters(java.io.DataOutputStream os)
                    throws java.io.IOException

Writes the parameters of the block to the given outputStream.

Specified by:

saveParameters in interface Block

Overrides:

saveParameters in class RecurrentCell

Parameters:

os - the outputstream to save the parameters to

Throws:

java.io.IOException - if an I/O error occurs

loadParameters

public void loadParameters(NDManager manager,
                           java.io.DataInputStream is)
                    throws java.io.IOException,
                           MalformedModelException

Loads the parameters from the given input stream.

Specified by:

loadParameters in interface Block

Overrides:

loadParameters in class RecurrentCell

Parameters:

manager - an NDManager to create the parameter arrays

is - the inputstream that stream the parameter values

Throws:

java.io.IOException - if an I/O error occurs

MalformedModelException - if the model file is corrupted or unsupported

builder

public static LSTM.Builder builder()

Creates a builder to build a LSTM.

Returns:

a new builder