org.deeplearning4j.datasets.iterator.parallel

Class FileSplitParallelDataSetIterator

java.lang.Object

org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

org.deeplearning4j.datasets.iterator.parallel.FileSplitParallelDataSetIterator

All Implemented Interfaces:

Serializable, Iterator<DataSet>, DataSetIterator, ParallelDataSetIterator

public class FileSplitParallelDataSetIterator
extends BaseParallelDataSetIterator

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected List<DataSetIterator>	asyncIterators
static String	DEFAULT_PATTERN

Fields inherited from class org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

allDepleted, counter, inequalityHandling, numProducers, producerAffinity, resetTracker, states

Constructor Summary

Constructors

Constructor and Description
FileSplitParallelDataSetIterator(@NonNull File rootFolder, @NonNull String pattern, @NonNull FileCallback callback)
FileSplitParallelDataSetIterator(@NonNull File rootFolder, @NonNull String pattern, @NonNull FileCallback callback, int numThreads)
FileSplitParallelDataSetIterator(@NonNull File rootFolder, @NonNull String pattern, @NonNull FileCallback callback, int numThreads, @NonNull InequalityHandling inequalityHandling)
FileSplitParallelDataSetIterator(@NonNull File rootFolder, @NonNull String pattern, @NonNull FileCallback callback, int numThreads, int bufferPerThread, @NonNull InequalityHandling inequalityHandling)

Method Summary

Modifier and Type	Method and Description
boolean	hasNextFor(int consumer)
DataSet	nextFor(int consumer)
protected void	reset(int consumer)

Methods inherited from class org.deeplearning4j.datasets.iterator.parallel.BaseParallelDataSetIterator

asyncSupported, attachThread, batch, getCurrentProducerIndex, getLabels, getPreProcessor, hasNext, hasNextFor, inputColumns, next, next, nextFor, remove, reset, resetSupported, setPreProcessor, stepForward, totalOutcomes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.Iterator

forEachRemaining

Field Detail

DEFAULT_PATTERN

public static final String DEFAULT_PATTERN

See Also:

Constant Field Values

asyncIterators

protected List<DataSetIterator> asyncIterators

Constructor Detail

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        @NonNull File rootFolder,
                                        @NonNull
                                        @NonNull String pattern,
                                        @NonNull
                                        @NonNull FileCallback callback)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        @NonNull File rootFolder,
                                        @NonNull
                                        @NonNull String pattern,
                                        @NonNull
                                        @NonNull FileCallback callback,
                                        int numThreads)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        @NonNull File rootFolder,
                                        @NonNull
                                        @NonNull String pattern,
                                        @NonNull
                                        @NonNull FileCallback callback,
                                        int numThreads,
                                        @NonNull
                                        @NonNull InequalityHandling inequalityHandling)

FileSplitParallelDataSetIterator

public FileSplitParallelDataSetIterator(@NonNull
                                        @NonNull File rootFolder,
                                        @NonNull
                                        @NonNull String pattern,
                                        @NonNull
                                        @NonNull FileCallback callback,
                                        int numThreads,
                                        int bufferPerThread,
                                        @NonNull
                                        @NonNull InequalityHandling inequalityHandling)

Method Detail

hasNextFor

public boolean hasNextFor(int consumer)

Specified by:

hasNextFor in interface ParallelDataSetIterator

Specified by:

hasNextFor in class BaseParallelDataSetIterator

nextFor

public DataSet nextFor(int consumer)

Specified by:

nextFor in interface ParallelDataSetIterator

Specified by:

nextFor in class BaseParallelDataSetIterator

reset

protected void reset(int consumer)

Specified by:

reset in class BaseParallelDataSetIterator