org.nd4j.linalg.api.ops.impl.transforms.custom

Class LayerNorm

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.DynamicCustomOp

org.nd4j.linalg.api.ops.impl.transforms.custom.LayerNorm

All Implemented Interfaces:

CustomOp

public class LayerNorm
extends DynamicCustomOp

Nested Class Summary

Nested classes/interfaces inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

DynamicCustomOp.DynamicCustomOpsBuilder

Field Summary

Fields inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

axis, bArguments, dArguments, iArguments, inplaceCall, inputArguments, outputArguments, outputVariables, tArguments

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, ownName, ownNameSetWithDefault, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
LayerNorm(@NonNull INDArray input, @NonNull INDArray gain, boolean channelsFirst, int... dimensions)
LayerNorm(INDArray input, INDArray gain, INDArray result, boolean channelsFirst, int... dimensions)
LayerNorm(INDArray input, INDArray gain, INDArray bias, INDArray result, boolean channelsFirst, int... dimensions)
LayerNorm(SameDiff sameDiff, SDVariable input, SDVariable gain, boolean channelsFirst, int... dimensions)
LayerNorm(@NonNull SameDiff sameDiff, @NonNull SDVariable input, @NonNull SDVariable gain, SDVariable bias, boolean channelsFirst, int... dimensions)

Method Summary

Modifier and Type	Method and Description
List<DataType>	calculateOutputDataTypes(List<DataType> dataTypes) Calculate the data types for the output arrays.
List<SDVariable>	doDiff(List<SDVariable> gradient) The actual implementation for automatic differentiation.
int	numOutputArguments()
String	onnxName() The opName of this function in onnx
String	opName() This method returns op opName as string
void	setDimensions(int[] dimensions)
String	tensorflowName() The opName of this function tensorflow

Methods inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

addBArgument, addDArgument, addIArgument, addIArgument, addInputArgument, addOutputArgument, addTArgument, assertValidForExecution, bArgs, builder, calculateOutputShape, calculateOutputShape, clearArrays, dArgs, getBArgument, getDescriptor, getIArgument, getInputArgument, getOutputArgument, getTArgument, iArgs, initFromOnnx, initFromTensorFlow, inputArguments, numBArguments, numDArguments, numIArguments, numInputArguments, numTArguments, opHash, opNum, opType, outputArguments, outputVariables, outputVariables, removeIArgument, removeInputArgument, removeOutputArgument, removeTArgument, setInputArgument, setInputArguments, setOutputArgument, tArgs, toString, wrapFilterNull, wrapOrNull, wrapOrNull

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, arg, argNames, args, attributeAdaptersForFunction, configFieldName, diff, dup, equals, getNumOutputs, getValue, hashCode, isConfigProperties, larg, mappingsForFunction, onnxNames, outputs, outputVariable, outputVariablesNames, propertiesForFunction, rarg, replaceArg, setInstanceId, setPropertiesForFunction, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.CustomOp

isInplaceCall

Constructor Detail

LayerNorm

public LayerNorm(@NonNull
                 @NonNull SameDiff sameDiff,
                 @NonNull
                 @NonNull SDVariable input,
                 @NonNull
                 @NonNull SDVariable gain,
                 SDVariable bias,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(SameDiff sameDiff,
                 SDVariable input,
                 SDVariable gain,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(INDArray input,
                 INDArray gain,
                 INDArray bias,
                 INDArray result,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(@NonNull
                 @NonNull INDArray input,
                 @NonNull
                 @NonNull INDArray gain,
                 boolean channelsFirst,
                 int... dimensions)

LayerNorm

public LayerNorm(INDArray input,
                 INDArray gain,
                 INDArray result,
                 boolean channelsFirst,
                 int... dimensions)

Method Detail

setDimensions

public void setDimensions(int[] dimensions)

opName

public String opName()

Description copied from class: DynamicCustomOp

This method returns op opName as string

Specified by:

opName in interface CustomOp

Overrides:

opName in class DynamicCustomOp

Returns:

tensorflowName

public String tensorflowName()

Description copied from class: DifferentialFunction

The opName of this function tensorflow

Overrides:

tensorflowName in class DynamicCustomOp

Returns:

onnxName

public String onnxName()

Description copied from class: DifferentialFunction

The opName of this function in onnx

Overrides:

onnxName in class DynamicCustomOp

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> gradient)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Overrides:

doDiff in class DynamicCustomOp

Returns:

calculateOutputDataTypes

public List<DataType> calculateOutputDataTypes(List<DataType> dataTypes)

Description copied from class: DifferentialFunction

Calculate the data types for the output arrays. Though datatypes can also be inferred from DifferentialFunction.calculateOutputShape(), this method differs in that it does not require the input arrays to be populated. This is important as it allows us to do greedy datatype inference for the entire net - even if arrays are not available.

Overrides:

calculateOutputDataTypes in class DifferentialFunction

Parameters:

dataTypes - The data types of the inputs

Returns:

The data types of the outputs

numOutputArguments

public int numOutputArguments()

Specified by:

numOutputArguments in interface CustomOp

Overrides:

numOutputArguments in class DynamicCustomOp