org.nd4j.linalg.api.ops.impl.transforms.custom

Class MultiHeadDotProductAttentionBp

java.lang.Object

org.nd4j.autodiff.functions.DifferentialFunction

org.nd4j.linalg.api.ops.DynamicCustomOp

org.nd4j.linalg.api.ops.impl.transforms.custom.MultiHeadDotProductAttentionBp

All Implemented Interfaces:

CustomOp

public class MultiHeadDotProductAttentionBp
extends DynamicCustomOp

Nested Class Summary

Nested classes/interfaces inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

DynamicCustomOp.DynamicCustomOpsBuilder

Field Summary

Fields inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

axis, bArguments, dArguments, iArguments, inplaceCall, inputArguments, outputArguments, outputVariables, tArguments

Fields inherited from class org.nd4j.autodiff.functions.DifferentialFunction

dimensions, extraArgs, inPlace, ownName, ownNameSetWithDefault, sameDiff, scalarValue

Constructor Summary

Constructors

Constructor and Description
MultiHeadDotProductAttentionBp(SameDiff sameDiff, SDVariable queries, SDVariable keys, SDVariable values, SDVariable Wq, SDVariable Wk, SDVariable Wv, SDVariable Wo, SDVariable eps, SDVariable mask, boolean scaled)

Method Summary

Modifier and Type	Method and Description
List<DataType>	calculateOutputDataTypes(List<DataType> dataTypes) Calculate the data types for the output arrays.
List<SDVariable>	doDiff(List<SDVariable> grad) The actual implementation for automatic differentiation.
String	opName() This method returns op opName as string

Methods inherited from class org.nd4j.linalg.api.ops.DynamicCustomOp

addBArgument, addDArgument, addIArgument, addIArgument, addInputArgument, addOutputArgument, addTArgument, assertValidForExecution, bArgs, builder, calculateOutputShape, calculateOutputShape, clearArrays, dArgs, getBArgument, getDescriptor, getIArgument, getInputArgument, getOutputArgument, getTArgument, iArgs, initFromOnnx, initFromTensorFlow, inputArguments, numBArguments, numDArguments, numIArguments, numInputArguments, numOutputArguments, numTArguments, onnxName, opHash, opNum, opType, outputArguments, outputVariables, outputVariables, removeIArgument, removeInputArgument, removeOutputArgument, removeTArgument, setInputArgument, setInputArguments, setOutputArgument, tArgs, tensorflowName, toString, wrapFilterNull, wrapOrNull, wrapOrNull

Methods inherited from class org.nd4j.autodiff.functions.DifferentialFunction

arg, arg, argNames, args, attributeAdaptersForFunction, configFieldName, diff, dup, equals, getNumOutputs, getValue, hashCode, isConfigProperties, larg, mappingsForFunction, onnxNames, outputs, outputVariable, outputVariablesNames, propertiesForFunction, rarg, replaceArg, setInstanceId, setPropertiesForFunction, setValueFor, tensorflowNames

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.api.ops.CustomOp

isInplaceCall

Constructor Detail

MultiHeadDotProductAttentionBp

public MultiHeadDotProductAttentionBp(SameDiff sameDiff,
                                      SDVariable queries,
                                      SDVariable keys,
                                      SDVariable values,
                                      SDVariable Wq,
                                      SDVariable Wk,
                                      SDVariable Wv,
                                      SDVariable Wo,
                                      SDVariable eps,
                                      SDVariable mask,
                                      boolean scaled)

Method Detail

opName

public String opName()

Description copied from class: DynamicCustomOp

This method returns op opName as string

Specified by:

opName in interface CustomOp

Overrides:

opName in class DynamicCustomOp

Returns:

doDiff

public List<SDVariable> doDiff(List<SDVariable> grad)

Description copied from class: DifferentialFunction

The actual implementation for automatic differentiation.

Overrides:

doDiff in class DynamicCustomOp

Returns:

calculateOutputDataTypes

public List<DataType> calculateOutputDataTypes(List<DataType> dataTypes)

Description copied from class: DifferentialFunction

Calculate the data types for the output arrays. Though datatypes can also be inferred from DifferentialFunction.calculateOutputShape(), this method differs in that it does not require the input arrays to be populated. This is important as it allows us to do greedy datatype inference for the entire net - even if arrays are not available.

Overrides:

calculateOutputDataTypes in class DifferentialFunction

Parameters:

dataTypes - The data types of the inputs

Returns:

The data types of the outputs