org.nd4j.linalg.dataset.api.preprocessor

Class AbstractMultiDataSetNormalizer<S extends NormalizerStats>

java.lang.Object

org.nd4j.linalg.dataset.api.preprocessor.AbstractNormalizer

org.nd4j.linalg.dataset.api.preprocessor.AbstractMultiDataSetNormalizer<S>

All Implemented Interfaces:

Serializable, MultiDataSetPreProcessor, MultiDataNormalization, Normalizer<MultiDataSet>

Direct Known Subclasses:

MultiNormalizerMinMaxScaler, MultiNormalizerStandardize

public abstract class AbstractMultiDataSetNormalizer<S extends NormalizerStats>
extends AbstractNormalizer
implements MultiDataNormalization

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected NormalizerStrategy<S>	strategy

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractMultiDataSetNormalizer()
protected	AbstractMultiDataSetNormalizer(NormalizerStrategy<S> strategy)

Method Summary

Modifier and Type	Method and Description
void	fit(@NonNull MultiDataSet dataSet) Fit a MultiDataSet (only compute based on the statistics from this MultiDataSet)
void	fit(@NonNull MultiDataSetIterator iterator) Fit an iterator
void	fitLabel(boolean fitLabels) Flag to specify if the labels/outputs in the dataset should be also normalized default value is false
protected List<S>	getFeatureStats()
protected S	getFeatureStats(int input)
protected List<S>	getLabelStats()
protected S	getLabelStats(int output)
protected boolean	isFit()
boolean	isFitLabel() Whether normalization for the labels is also enabled.
protected abstract NormalizerStats.Builder	newBuilder()
int	numInputs() Get the number of inputs
int	numOutputs() Get the number of outputs
void	preProcess(@NonNull MultiDataSet toPreProcess) Pre process a MultiDataSet
void	revert(@NonNull MultiDataSet data) Revert the data to what it was before transform
void	revertFeatures(@NonNull INDArray[] features) Undo (revert) the normalization applied by this normalizer to the features arrays
void	revertFeatures(@NonNull INDArray[] features, INDArray[] maskArrays) Undo (revert) the normalization applied by this normalizer to the features arrays
void	revertFeatures(@NonNull INDArray features, INDArray mask, int input) Undo (revert) the normalization applied by this normalizer to a specific features array.
void	revertLabels(INDArray[] labels) Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array.
void	revertLabels(@NonNull INDArray[] labels, INDArray[] labelsMask) Undo (revert) the normalization applied by this normalizer to the labels arrays.
void	revertLabels(@NonNull INDArray labels, INDArray mask, int output) Undo (revert) the normalization applied by this normalizer to a specific labels array.
void	transform(@NonNull MultiDataSet toPreProcess) Transform the dataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.dataset.api.preprocessor.Normalizer

getType

Field Detail

strategy

protected NormalizerStrategy<S extends NormalizerStats> strategy

Constructor Detail

AbstractMultiDataSetNormalizer

protected AbstractMultiDataSetNormalizer()

AbstractMultiDataSetNormalizer

protected AbstractMultiDataSetNormalizer(NormalizerStrategy<S> strategy)

Method Detail

fitLabel

public void fitLabel(boolean fitLabels)

Flag to specify if the labels/outputs in the dataset should be also normalized default value is false

Parameters:

fitLabels -

isFitLabel

public boolean isFitLabel()

Whether normalization for the labels is also enabled. Most commonly used for regression, not classification.

Returns:

True if labels will be

isFit

protected boolean isFit()

Specified by:

isFit in class AbstractNormalizer

getFeatureStats

protected S getFeatureStats(int input)

getFeatureStats

protected List<S> getFeatureStats()

getLabelStats

protected S getLabelStats(int output)

getLabelStats

protected List<S> getLabelStats()

fit

public void fit(@NonNull
                @NonNull MultiDataSet dataSet)

Fit a MultiDataSet (only compute based on the statistics from this MultiDataSet)

Specified by:

fit in interface Normalizer<MultiDataSet>

Parameters:

dataSet - the dataset to compute on

fit

public void fit(@NonNull
                @NonNull MultiDataSetIterator iterator)

Fit an iterator

Specified by:

fit in interface MultiDataNormalization

Parameters:

iterator - for the data to iterate over

newBuilder

protected abstract NormalizerStats.Builder newBuilder()

transform

public void transform(@NonNull
                      @NonNull MultiDataSet toPreProcess)

Description copied from interface: Normalizer

Transform the dataset

Specified by:

transform in interface Normalizer<MultiDataSet>

Parameters:

toPreProcess - the dataset to re process

preProcess

public void preProcess(@NonNull
                       @NonNull MultiDataSet toPreProcess)

Pre process a MultiDataSet

Specified by:

preProcess in interface MultiDataSetPreProcessor

Specified by:

preProcess in interface MultiDataNormalization

Parameters:

toPreProcess - the data set to pre process

revert

public void revert(@NonNull
                   @NonNull MultiDataSet data)

Revert the data to what it was before transform

Specified by:

revert in interface Normalizer<MultiDataSet>

Parameters:

data - the dataset to revert back

revertFeatures

public void revertFeatures(@NonNull
                           @NonNull INDArray[] features)

Undo (revert) the normalization applied by this normalizer to the features arrays

Specified by:

revertFeatures in interface MultiDataNormalization

Parameters:

features - Features to revert the normalization on

revertFeatures

public void revertFeatures(@NonNull
                           @NonNull INDArray[] features,
                           INDArray[] maskArrays)

Undo (revert) the normalization applied by this normalizer to the features arrays

Specified by:

revertFeatures in interface MultiDataNormalization

Parameters:

features - Features to revert the normalization on

revertFeatures

public void revertFeatures(@NonNull
                           @NonNull INDArray features,
                           INDArray mask,
                           int input)

Undo (revert) the normalization applied by this normalizer to a specific features array. If labels normalization is disabled (i.e., isFitLabel() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Parameters:

features - features arrays to revert the normalization on

input - the index of the array to revert

revertLabels

public void revertLabels(INDArray[] labels)

Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array. If labels normalization is disabled (i.e., isFitLabel() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Specified by:

revertLabels in interface MultiDataNormalization

Parameters:

labels - Labels array to revert the normalization on

revertLabels

public void revertLabels(@NonNull
                         @NonNull INDArray[] labels,
                         INDArray[] labelsMask)

Undo (revert) the normalization applied by this normalizer to the labels arrays. If labels normalization is disabled (i.e., isFitLabel() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Specified by:

revertLabels in interface MultiDataNormalization

Parameters:

labels - Labels arrays to revert the normalization on

labelsMask - Labels mask array (may be null)

revertLabels

public void revertLabels(@NonNull
                         @NonNull INDArray labels,
                         INDArray mask,
                         int output)

Undo (revert) the normalization applied by this normalizer to a specific labels array. If labels normalization is disabled (i.e., isFitLabel() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Parameters:

labels - Labels arrays to revert the normalization on

output - the index of the array to revert

numInputs

public int numInputs()

Get the number of inputs

numOutputs

public int numOutputs()

Get the number of outputs