org.nd4j.linalg.dataset.api.preprocessor

Class AbstractDataSetNormalizer<S extends NormalizerStats>

java.lang.Object

org.nd4j.linalg.dataset.api.preprocessor.AbstractNormalizer

org.nd4j.linalg.dataset.api.preprocessor.AbstractDataSetNormalizer<S>

All Implemented Interfaces:

Serializable, DataSetPreProcessor, DataNormalization, Normalizer<DataSet>

Direct Known Subclasses:

NormalizerMinMaxScaler, NormalizerStandardize

public abstract class AbstractDataSetNormalizer<S extends NormalizerStats>
extends AbstractNormalizer
implements DataNormalization

Abstract base class for normalizers that act upon DataSet instances or iterators

Author:

Ede Meijer

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected NormalizerStrategy<S>	strategy

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractDataSetNormalizer(NormalizerStrategy<S> strategy)

Method Summary

Modifier and Type	Method and Description
void	fit(DataSet dataSet) Fit a dataset (only compute based on the statistics from this dataset)
void	fit(DataSetIterator iterator) Fit the given model
void	fitLabel(boolean fitLabels) Flag to specify if the labels/outputs in the dataset should be also normalized default value is false
protected S	getFeatureStats()
protected S	getLabelStats()
protected boolean	isFit()
boolean	isFitLabel() Whether normalization for the labels is also enabled.
protected abstract NormalizerStats.Builder	newBuilder()
void	preProcess(DataSet toPreProcess) Pre process a dataset
void	revert(DataSet data) Revert the data to what it was before transform
void	revertFeatures(INDArray features) Undo (revert) the normalization applied by this DataNormalization instance to the specified features array
void	revertFeatures(INDArray features, INDArray featuresMask) Undo (revert) the normalization applied by this DataNormalization instance to the specified features array
void	revertLabels(INDArray labels) Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array.
void	revertLabels(INDArray labels, INDArray labelsMask) Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array.
void	transform(DataSet toPreProcess) Transform the given dataset
void	transform(INDArray features) Transform the given INDArray
void	transform(INDArray features, INDArray featuresMask) Transform the features, with an optional mask array
void	transformLabel(INDArray label) Transform the labels.
void	transformLabel(INDArray label, INDArray labelsMask) Transform the labels.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.nd4j.linalg.dataset.api.preprocessor.Normalizer

getType

Field Detail

strategy

protected NormalizerStrategy<S extends NormalizerStats> strategy

Constructor Detail

AbstractDataSetNormalizer

protected AbstractDataSetNormalizer(NormalizerStrategy<S> strategy)

Method Detail

fitLabel

public void fitLabel(boolean fitLabels)

Flag to specify if the labels/outputs in the dataset should be also normalized default value is false

Specified by:

fitLabel in interface DataNormalization

Parameters:

fitLabels -

isFitLabel

public boolean isFitLabel()

Whether normalization for the labels is also enabled. Most commonly used for regression, not classification.

Specified by:

isFitLabel in interface DataNormalization

Returns:

True if labels will be

fit

public void fit(DataSet dataSet)

Fit a dataset (only compute based on the statistics from this dataset)

Specified by:

fit in interface Normalizer<DataSet>

Parameters:

dataSet - the dataset to compute on

getFeatureStats

protected S getFeatureStats()

getLabelStats

protected S getLabelStats()

isFit

protected boolean isFit()

Specified by:

isFit in class AbstractNormalizer

fit

public void fit(DataSetIterator iterator)

Fit the given model

Specified by:

fit in interface DataNormalization

Parameters:

iterator - for the data to iterate over

newBuilder

protected abstract NormalizerStats.Builder newBuilder()

preProcess

public void preProcess(@NonNull
                       DataSet toPreProcess)

Pre process a dataset

Specified by:

preProcess in interface DataSetPreProcessor

Specified by:

preProcess in interface DataNormalization

Parameters:

toPreProcess - the data set to pre process

transform

public void transform(DataSet toPreProcess)

Transform the given dataset

Specified by:

transform in interface Normalizer<DataSet>

Parameters:

toPreProcess -

transform

public void transform(INDArray features)

Transform the given INDArray

Specified by:

transform in interface DataNormalization

Parameters:

features -

transform

public void transform(INDArray features,
                      INDArray featuresMask)

Description copied from interface: DataNormalization

Transform the features, with an optional mask array

Specified by:

transform in interface DataNormalization

Parameters:

features - the features to pre process

featuresMask - the mask array to pre process

transformLabel

public void transformLabel(INDArray label)

Transform the labels. If isFitLabel() == false, this is a no-op

Specified by:

transformLabel in interface DataNormalization

transformLabel

public void transformLabel(INDArray label,
                           INDArray labelsMask)

Description copied from interface: DataNormalization

Transform the labels. If DataNormalization.isFitLabel() == false, this is a no-op

Specified by:

transformLabel in interface DataNormalization

revertFeatures

public void revertFeatures(INDArray features)

Description copied from interface: DataNormalization

Undo (revert) the normalization applied by this DataNormalization instance to the specified features array

Specified by:

revertFeatures in interface DataNormalization

Parameters:

features - Features to revert the normalization on

revertFeatures

public void revertFeatures(INDArray features,
                           INDArray featuresMask)

Description copied from interface: DataNormalization

Undo (revert) the normalization applied by this DataNormalization instance to the specified features array

Specified by:

revertFeatures in interface DataNormalization

Parameters:

features - Features to revert the normalization on

revertLabels

public void revertLabels(INDArray labels)

Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array. If labels normalization is disabled (i.e., isFitLabel() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Specified by:

revertLabels in interface DataNormalization

Parameters:

labels - Labels array to revert the normalization on

revertLabels

public void revertLabels(INDArray labels,
                         INDArray labelsMask)

Description copied from interface: DataNormalization

Undo (revert) the normalization applied by this DataNormalization instance to the specified labels array. If labels normalization is disabled (i.e., #isFitLabels() == false) then this is a no-op. Can also be used to undo normalization for network output arrays, in the case of regression.

Specified by:

revertLabels in interface DataNormalization

Parameters:

labels - Labels array to revert the normalization on

labelsMask - Labels mask array (may be null)

revert

public void revert(DataSet data)

Revert the data to what it was before transform

Specified by:

revert in interface Normalizer<DataSet>

Parameters:

data - the dataset to revert back