CifarLoader (datavec-data-image 0.7.2 API)

java.lang.Object
- org.datavec.image.loader.BaseImageLoader
- - org.datavec.image.loader.NativeImageLoader
  - - org.datavec.image.loader.CifarLoader

All Implemented Interfaces:

Serializable
```
public class CifarLoader
extends NativeImageLoader
implements Serializable
```
CifarLoader is loader specific for the Cifar10 dataset Reference: Learning Multiple Layers of Features from Tiny Images, Alex Krizhevsky, 2009. There is a special preProcessor used to normalize the dataset based on Sergey Zagoruyko example https://github.com/szagoruyko/cifar.torch

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`static int`	`BYTEFILELEN`
`protected static int`	`channels`
`static int`	`CHANNELS`
`static Map<String,String>`	`cifarDataMap`
`static String`	`dataBinFile`
`static String`	`dataBinUrl`
`protected int`	`fileNum`
`static File`	`fullDir`
`protected static int`	`height`
`static int`	`HEIGHT`
`protected static List<org.nd4j.linalg.dataset.DataSet>`	`inputBatched`
`protected static InputStream`	`inputStream`
`protected static String`	`labelFileName`
`protected static List<String>`	`labels`
`protected org.nd4j.linalg.dataset.DataSet`	`loadDS`
`protected int`	`loadDSIndex`
`static String`	`localDir`
`protected boolean`	`meanStdStored`
`static File`	`meanVarPath`
`static int`	`NUM_LABELS`
`static int`	`NUM_TEST_IMAGES`
`static int`	`NUM_TRAIN_IMAGES`
`protected int`	`numExamples`
`protected static int`	`numToConvertDS`
`protected static long`	`seed`
`protected static boolean`	`shuffle`
`static String`	`TESTFILENAME`
`protected static String`	`testFilesSerialized`
`protected static InputStream`	`testInputStream`
`protected static boolean`	`train`
`static String[]`	`TRAINFILENAMES`
`protected static String`	`trainFilesSerialized`
`protected static InputStream`	`trainInputStream`
`protected double`	`uMean`
`static boolean`	`useSpecialPreProcessCifar`
`protected double`	`uStd`
`protected double`	`vMean`
`protected double`	`vStd`
`protected static int`	`width`
`static int`	`WIDTH`

Fields inherited from class org.datavec.image.loader.NativeImageLoader
ALLOWED_FORMATS

Fields inherited from class org.datavec.image.loader.BaseImageLoader
BASE_DIR, centerCropIfNeeded, imageTransform, log, rng

Constructor Summary

Constructors
Constructor and Description
`CifarLoader()`
`CifarLoader(boolean train)`
`CifarLoader(boolean train, File fullPath)`
`CifarLoader(int height, int width, int channels, boolean train, boolean useSpecialPreProcessCifar)`
`CifarLoader(int height, int width, int channels, ImageTransform imgTransform, boolean train, boolean useSpecialPreProcessCifar)`
`CifarLoader(int height, int width, int channels, ImageTransform imgTransform, boolean train, boolean useSpecialPreProcessCifar, boolean shuffle)`
`CifarLoader(int height, int width, int channels, ImageTransform imgTransform, boolean train, boolean useSpecialPreProcessCifar, File fullPath, long seed, boolean shuffle)`

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`org.nd4j.linalg.api.ndarray.INDArray`	`asMatrix(File f)`
`org.nd4j.linalg.api.ndarray.INDArray`	`asMatrix(InputStream inputStream)`
`org.nd4j.linalg.api.ndarray.INDArray`	`asRowVector(File f)` Convert a file to a row vector
`org.nd4j.linalg.api.ndarray.INDArray`	`asRowVector(InputStream inputStream)`
`boolean`	`cifarRawFilesExist()`
`org.bytedeco.javacpp.opencv_core.Mat`	`convertCifar(org.bytedeco.javacpp.opencv_core.Mat orgImage)` Preprocess and store cifar based on successful Torch approach by Sergey Zagoruyko Reference: https://github.com/szagoruyko/cifar.torch
`org.nd4j.linalg.dataset.DataSet`	`convertDataSet(int num)`
`org.datavec.api.berkeley.Pair<org.nd4j.linalg.api.ndarray.INDArray,org.bytedeco.javacpp.opencv_core.Mat>`	`convertMat(byte[] byteFeature)`
`void`	`generateMaps()`
`InputStream`	`getInputStream()`
`List<String>`	`getLabels()`
`void`	`load()`
`org.nd4j.linalg.dataset.DataSet`	`next(int batchSize)`
`org.nd4j.linalg.dataset.DataSet`	`next(int batchSize, int exampleNum)`
`void`	`normalizeCifar(File fileName)` Normalize and store cifar based on successful Torch approach by Sergey Zagoruyko Reference: https://github.com/szagoruyko/cifar.torch
`void`	`reset()`
`void`	`setInputStream()`
`void`	`test()`
`void`	`train()`
`double`	`varManual(org.nd4j.linalg.api.ndarray.INDArray x, double mean)`

Methods inherited from class org.datavec.image.loader.NativeImageLoader
asMatrix, asMatrix, asRowVector, asRowVector, centerCropIfNeeded, getAllowedFormats, scalingIfNeed, scalingIfNeed

Methods inherited from class org.datavec.image.loader.BaseImageLoader
downloadAndUntar

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

NUM_TRAIN_IMAGES
```
public static final int NUM_TRAIN_IMAGES
```
See Also:

Constant Field Values

NUM_TEST_IMAGES
```
public static final int NUM_TEST_IMAGES
```
See Also:

Constant Field Values

NUM_LABELS
```
public static final int NUM_LABELS
```
See Also:

Constant Field Values

HEIGHT
```
public static final int HEIGHT
```
See Also:

Constant Field Values

WIDTH
```
public static final int WIDTH
```
See Also:

Constant Field Values

CHANNELS
```
public static final int CHANNELS
```
See Also:

Constant Field Values

BYTEFILELEN
```
public static final int BYTEFILELEN
```
See Also:

Constant Field Values

dataBinUrl
```
public static String dataBinUrl
```

localDir
```
public static String localDir
```

dataBinFile
```
public static String dataBinFile
```

fullDir
```
public static File fullDir
```

meanVarPath
```
public static File meanVarPath
```

labelFileName
```
protected static String labelFileName
```

inputStream

protected static InputStream inputStream

trainInputStream

protected static InputStream trainInputStream

testInputStream

protected static InputStream testInputStream

inputBatched

protected static List<org.nd4j.linalg.dataset.DataSet> inputBatched

labels
```
protected static List<String> labels
```

TRAINFILENAMES
```
public static String[] TRAINFILENAMES
```

TESTFILENAME
```
public static String TESTFILENAME
```

trainFilesSerialized

protected static String trainFilesSerialized

testFilesSerialized

protected static String testFilesSerialized

train
```
protected static boolean train
```

useSpecialPreProcessCifar

public static boolean useSpecialPreProcessCifar

cifarDataMap

public static Map<String,String> cifarDataMap

height
```
protected static int height
```

width
```
protected static int width
```

channels
```
protected static int channels
```

seed
```
protected static long seed
```

shuffle
```
protected static boolean shuffle
```

numExamples
```
protected int numExamples
```

numToConvertDS
```
protected static int numToConvertDS
```

uMean
```
protected double uMean
```

uStd
```
protected double uStd
```

vMean
```
protected double vMean
```

vStd
```
protected double vStd
```

meanStdStored
```
protected boolean meanStdStored
```

loadDSIndex
```
protected int loadDSIndex
```

loadDS

protected org.nd4j.linalg.dataset.DataSet loadDS

fileNum
```
protected int fileNum
```

Constructor Detail

CifarLoader
```
public CifarLoader()
```

CifarLoader
```
public CifarLoader(boolean train)
```

CifarLoader

public CifarLoader(boolean train,
                   File fullPath)

CifarLoader

public CifarLoader(int height,
                   int width,
                   int channels,
                   boolean train,
                   boolean useSpecialPreProcessCifar)

CifarLoader

public CifarLoader(int height,
                   int width,
                   int channels,
                   ImageTransform imgTransform,
                   boolean train,
                   boolean useSpecialPreProcessCifar)

CifarLoader

public CifarLoader(int height,
                   int width,
                   int channels,
                   ImageTransform imgTransform,
                   boolean train,
                   boolean useSpecialPreProcessCifar,
                   boolean shuffle)

CifarLoader

public CifarLoader(int height,
                   int width,
                   int channels,
                   ImageTransform imgTransform,
                   boolean train,
                   boolean useSpecialPreProcessCifar,
                   File fullPath,
                   long seed,
                   boolean shuffle)

Method Detail

asRowVector
```
public org.nd4j.linalg.api.ndarray.INDArray asRowVector(File f)
                                                 throws IOException
```
Description copied from class: NativeImageLoader

Convert a file to a row vector

Overrides:

asRowVector in class NativeImageLoader

Parameters:

f - the image to convert

Returns:

the flattened image

Throws:

IOException

asRowVector

public org.nd4j.linalg.api.ndarray.INDArray asRowVector(InputStream inputStream)
                                                 throws IOException

Overrides:: asRowVector in class NativeImageLoader
Throws:: IOException

asMatrix

public org.nd4j.linalg.api.ndarray.INDArray asMatrix(File f)
                                              throws IOException

Overrides:: asMatrix in class NativeImageLoader
Throws:: IOException

asMatrix

public org.nd4j.linalg.api.ndarray.INDArray asMatrix(InputStream inputStream)
                                              throws IOException

Overrides:: asMatrix in class NativeImageLoader
Throws:: IOException

generateMaps
```
public void generateMaps()
```

load
```
public void load()
```

cifarRawFilesExist
```
public boolean cifarRawFilesExist()
```

convertCifar
```
public org.bytedeco.javacpp.opencv_core.Mat convertCifar(org.bytedeco.javacpp.opencv_core.Mat orgImage)
```
Preprocess and store cifar based on successful Torch approach by Sergey Zagoruyko Reference: https://github.com/szagoruyko/cifar.torch

normalizeCifar
```
public void normalizeCifar(File fileName)
```
Normalize and store cifar based on successful Torch approach by Sergey Zagoruyko Reference: https://github.com/szagoruyko/cifar.torch

convertMat

public org.datavec.api.berkeley.Pair<org.nd4j.linalg.api.ndarray.INDArray,org.bytedeco.javacpp.opencv_core.Mat> convertMat(byte[] byteFeature)

convertDataSet

public org.nd4j.linalg.dataset.DataSet convertDataSet(int num)

varManual

public double varManual(org.nd4j.linalg.api.ndarray.INDArray x,
                        double mean)

public org.nd4j.linalg.dataset.DataSet next(int batchSize)

public org.nd4j.linalg.dataset.DataSet next(int batchSize,
                                            int exampleNum)

getInputStream
```
public InputStream getInputStream()
```

setInputStream
```
public void setInputStream()
```

getLabels
```
public List<String> getLabels()
```

reset
```
public void reset()
```

train
```
public void train()
```

test
```
public void test()
```

Class CifarLoader

Field Summary

Fields inherited from class org.datavec.image.loader.NativeImageLoader

Fields inherited from class org.datavec.image.loader.BaseImageLoader

Constructor Summary

Method Summary

Methods inherited from class org.datavec.image.loader.NativeImageLoader

Methods inherited from class org.datavec.image.loader.BaseImageLoader

Methods inherited from class java.lang.Object

Field Detail

NUM_TRAIN_IMAGES

NUM_TEST_IMAGES

NUM_LABELS

HEIGHT

WIDTH

CHANNELS

BYTEFILELEN

dataBinUrl

localDir

dataBinFile

fullDir

meanVarPath

labelFileName

inputStream

trainInputStream

testInputStream

inputBatched

labels

TRAINFILENAMES

TESTFILENAME

trainFilesSerialized

testFilesSerialized

train

useSpecialPreProcessCifar

cifarDataMap

height

width

channels

seed

shuffle

numExamples

numToConvertDS

uMean

uStd

vMean

vStd

meanStdStored

loadDSIndex

loadDS

fileNum

Constructor Detail

CifarLoader

CifarLoader

CifarLoader

CifarLoader

CifarLoader

CifarLoader

CifarLoader

Method Detail

asRowVector

asRowVector

asMatrix

asMatrix

generateMaps

load

cifarRawFilesExist

convertCifar

normalizeCifar

convertMat

convertDataSet

varManual

next

next

getInputStream

setInputStream

getLabels

reset

train

test