ai.djl.basicdataset

Class AmesRandomAccess

java.lang.Object

ai.djl.training.dataset.RandomAccessDataset

ai.djl.basicdataset.AmesRandomAccess

All Implemented Interfaces:

ai.djl.training.dataset.Dataset

public class AmesRandomAccess
extends ai.djl.training.dataset.RandomAccessDataset

Ames house pricing dataset from https://www.kaggle.com/c/house-prices-advanced-regression-techniques/data.

80 features

Training Set: 1460 Records

Test Set: 1459 Records

Can enable/disable features Set one hot vector for categorical variables

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AmesRandomAccess.Builder A builder to construct a AmesRandomAccess.
static class	AmesRandomAccess.FeatureType An enum represent data type of feature.

Nested classes/interfaces inherited from class ai.djl.training.dataset.RandomAccessDataset

ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T extends ai.djl.training.dataset.RandomAccessDataset.BaseBuilder>

Nested classes/interfaces inherited from interface ai.djl.training.dataset.Dataset

ai.djl.training.dataset.Dataset.Usage

Field Summary

Fields inherited from class ai.djl.training.dataset.RandomAccessDataset

dataBatchifier, device, executor, labelBatchifier, limit, pipeline, prefetchNumber, sampler, targetPipeline

Method Summary

Modifier and Type	Method and Description
void	addAllFeatures() Move all non set features to be used.
void	addFeature(java.lang.String feature) Adds a feature if it exists.
void	addFeature(java.lang.String feature, AmesRandomAccess.FeatureType type) Adds a feature of the given type to the used features set.
protected long	availableSize()
static AmesRandomAccess.Builder	builder() Creates a builder to build a AmesRandomAccess.
ai.djl.training.dataset.Record	get(ai.djl.ndarray.NDManager manager, long index)
java.util.Set<java.lang.String>	getCategoricalFeatures() Get categorical feature set.
org.apache.commons.csv.CSVRecord	getCSVRecord(int index) Returns the CSVRecord for a given index.
java.util.Set<java.lang.String>	getDisabledFeatures() Get disabled feature set.
java.util.Set<java.lang.String>	getEnabledFeatures() Get enabled feature set.
int	getFeatureArraySize() Returns the size of the feature array(column count).
ai.djl.ndarray.NDArray	getFeatureNDArray(ai.djl.ndarray.NDManager manager, int index) Return the NDArray at index 'index' with the set features.
AmesRandomAccess.FeatureType	getFeatureType(java.lang.String feature) Returns the feature type.
float[]	getLabel(int index) Returns the label value for a given index.
float[]	getValueFloat(org.apache.commons.csv.CSVRecord record, java.lang.String feature) Returns the float value of the record's feature Changes based on categorical or numeric and for categorical if 1 hot encode is enabled.
void	prepare(ai.djl.util.Progress progress)
void	removeAllFeatures() Move all currently set features to available.
void	removeFeature(java.lang.String feature) Removes a feature from the active set if it exists.
void	setFeatureType(java.lang.String feature, AmesRandomAccess.FeatureType type) Sets a feature's type.
void	setLabel(java.lang.String feature) Sets the label(y).
void	setOneHotEncode(java.lang.String feature, boolean enable) Sets a feature's oneHotEncode status if it is categorical.

Methods inherited from class ai.djl.training.dataset.RandomAccessDataset

getData, getData, randomSplit, size

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ai.djl.training.dataset.Dataset

prepare

Method Detail

builder

public static AmesRandomAccess.Builder builder()

Creates a builder to build a AmesRandomAccess.

Returns:

a new builder

setLabel

public void setLabel(java.lang.String feature)

Sets the label(y).

Parameters:

feature - feature to set as the label

getEnabledFeatures

public java.util.Set<java.lang.String> getEnabledFeatures()

Get enabled feature set.

Returns:

set of string features

getDisabledFeatures

public java.util.Set<java.lang.String> getDisabledFeatures()

Get disabled feature set.

Returns:

set of disabled features

getCategoricalFeatures

public java.util.Set<java.lang.String> getCategoricalFeatures()

Get categorical feature set.

Returns:

set of categorical features.

getLabel

public float[] getLabel(int index)

Returns the label value for a given index.

Parameters:

index - label index

Returns:

label value as a float wrapped in a float array

get

public ai.djl.training.dataset.Record get(ai.djl.ndarray.NDManager manager,
                                          long index)

Specified by:

get in class ai.djl.training.dataset.RandomAccessDataset

getCSVRecord

public org.apache.commons.csv.CSVRecord getCSVRecord(int index)

Returns the CSVRecord for a given index.

Parameters:

index - index of record

Returns:

CSVRecord for given index

setOneHotEncode

public void setOneHotEncode(java.lang.String feature,
                            boolean enable)

Sets a feature's oneHotEncode status if it is categorical.

Parameters:

feature - feature to be one hot encoded

enable - Enable/Disable oneHotEncoding

getValueFloat

public float[] getValueFloat(org.apache.commons.csv.CSVRecord record,
                             java.lang.String feature)

Returns the float value of the record's feature Changes based on categorical or numeric and for categorical if 1 hot encode is enabled.

Parameters:

record - The CSVRecord to get the feature from

feature - The feature value to be acquired

Returns:

feature value as a float wrapped in a float array

getFeatureArraySize

public int getFeatureArraySize()

Returns the size of the feature array(column count).

Returns:

feature array size

getFeatureNDArray

public ai.djl.ndarray.NDArray getFeatureNDArray(ai.djl.ndarray.NDManager manager,
                                                int index)

Return the NDArray at index 'index' with the set features.

Parameters:

manager - NDManager to maintain created NDArray

index - Index of wanted NDArray

Returns:

NDArray of enabled features

removeAllFeatures

public void removeAllFeatures()

Move all currently set features to available.

addAllFeatures

public void addAllFeatures()

Move all non set features to be used.

addFeature

public void addFeature(java.lang.String feature,
                       AmesRandomAccess.FeatureType type)

Adds a feature of the given type to the used features set.

Parameters:

feature - feature to be added

type - type of feature

addFeature

public void addFeature(java.lang.String feature)

Adds a feature if it exists.

Parameters:

feature - feature to be enabled

setFeatureType

public void setFeatureType(java.lang.String feature,
                           AmesRandomAccess.FeatureType type)

Sets a feature's type.

Parameters:

feature - feature whose type is to be chagned

type - type for feature to be set to

removeFeature

public void removeFeature(java.lang.String feature)

Removes a feature from the active set if it exists.

Parameters:

feature - feature to be disabled

getFeatureType

public AmesRandomAccess.FeatureType getFeatureType(java.lang.String feature)

Returns the feature type.

Parameters:

feature - feature whose type we will get

Returns:

type of feature

prepare

public void prepare(ai.djl.util.Progress progress)
             throws java.io.IOException

Throws:

java.io.IOException

availableSize

protected long availableSize()

Specified by:

availableSize in class ai.djl.training.dataset.RandomAccessDataset