ai.djl.nn.core

Class Prelu

java.lang.Object

ai.djl.nn.AbstractBlock

ai.djl.nn.core.Prelu

All Implemented Interfaces:

Block

public class Prelu
extends AbstractBlock

Applies Leaky Parametric ReLU activation element-wise to the input.

Leaky ReLUs attempt to fix the 'dying ReLU' problem by allowing a small slope when the input is negative and has a slope of one when input is positive. This is defined by \(y= x \gt 0 ? x : slope * x\).

Parametric ReLU is a Leaky ReLU in which the slope is learnt during training.

Field Summary

Fields inherited from class ai.djl.nn.AbstractBlock

children, inputNames, inputShapes, parameters, version

Constructor Summary

Constructors

Constructor and Description
Prelu() Creates a Parametric ReLU Block.

Method Summary

Modifier and Type	Method and Description
protected NDList	forwardInternal(ParameterStore parameterStore, NDList inputs, boolean training, ai.djl.util.PairList<java.lang.String,java.lang.Object> params) A helper for Block.forward(ParameterStore, NDList, boolean, PairList) after initialization.
Shape[]	getOutputShapes(Shape[] inputs) Returns the expected output shapes of the block for the specified input shapes.
void	loadMetadata(byte loadVersion, java.io.DataInputStream is) Overwrite this to load additional metadata with the parameter values.
static NDList	prelu(NDArray input, NDArray alpha) Applies a Prelu activation on the input NDArray.

Methods inherited from class ai.djl.nn.AbstractBlock

addChildBlock, addParameter, beforeInitialize, cast, clear, describeInput, forward, forward, forwardInternal, getChildren, getDirectParameters, getParameters, initialize, initializeChildBlocks, isInitialized, loadParameters, prepare, readInputShapes, saveInputShapes, saveMetadata, saveParameters, setInitializer, setInitializer, setInitializer, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface ai.djl.nn.Block

forward, validateLayout

Constructor Detail

Prelu

public Prelu()

Creates a Parametric ReLU Block.

Method Detail

forwardInternal

protected NDList forwardInternal(ParameterStore parameterStore,
                                 NDList inputs,
                                 boolean training,
                                 ai.djl.util.PairList<java.lang.String,java.lang.Object> params)

A helper for Block.forward(ParameterStore, NDList, boolean, PairList) after initialization.

Specified by:

forwardInternal in class AbstractBlock

Parameters:

parameterStore - the parameter store

inputs - the input NDList

training - true for a training forward pass

params - optional parameters

Returns:

the output of the forward pass

getOutputShapes

public Shape[] getOutputShapes(Shape[] inputs)

Returns the expected output shapes of the block for the specified input shapes.

Parameters:

inputs - the shapes of the inputs

Returns:

the expected output shapes of the block

loadMetadata

public void loadMetadata(byte loadVersion,
                         java.io.DataInputStream is)
                  throws java.io.IOException,
                         MalformedModelException

Overwrite this to load additional metadata with the parameter values.

If you overwrite AbstractBlock.saveMetadata(DataOutputStream) or need to provide backward compatibility to older binary formats, you prabably need to overwrite this. This default implementation checks if the version number fits, if not it throws an MalformedModelException. After that it restores the input shapes.

Overrides:

loadMetadata in class AbstractBlock

Parameters:

loadVersion - the version used for loading this metadata.

is - the input stream we are loading from

Throws:

java.io.IOException - loading failed

MalformedModelException - data can be loaded but has wrong format

prelu

public static NDList prelu(NDArray input,
                           NDArray alpha)

Applies a Prelu activation on the input NDArray.

Prelu is defined as \(y = max(0,x) + alpha * min(0, x) \) where alpha is learnable parameter

Parameters:

input - the input NDArray

alpha - learnable parameter

Returns:

the NDArray after applying Prelu activation