t

com.intel.analytics.bigdl.dataset

AbstractDataSet

trait AbstractDataSet[D, DataSequence] extends AnyRef

A set of data which is used in the model optimization process. The dataset can be access in a random data sample sequence. In the training process, the data sequence is a looped endless sequence. While in the validation process, the data sequence is a limited length sequence. User can use the data() method to get the data sequence.

The sequence of the data is not fixed. It can be changed by the shuffle() method.

User can create a dataset from a RDD, an array and a folder, etc. The DataSet object provides many factory methods.

D

Data type

DataSequence

Represent a sequence of data

Linear Supertypes
AnyRef, Any
Known Subclasses
CachedDistriDataSet, DistributedDataSet, LocalArrayDataSet, LocalDataSet
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. AbstractDataSet
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def data(train: Boolean): DataSequence

    Get a sequence of data

    Get a sequence of data

    train

    if the data is used in train. If yes, the data sequence is a looped endless sequence, or it has a limited length.

    returns

    data sequence

  2. abstract def shuffle(): Unit

    Change the order of the data sequence from the data set

  3. abstract def size(): Long

    Total size of the data set

  4. abstract def transform[C](transformer: Transformer[D, C])(implicit arg0: ClassTag[C]): DataSet[C]

    Helper function to transform the data type in the data set.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. def ->[C](transformer: Transformer[D, C])(implicit arg0: ClassTag[C]): DataSet[C]

    Helper function to transform the data type in the data set.

  4. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  5. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  6. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  16. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  17. def toDistributed(): DistributedDataSet[D]

    Convert current DataSet to a distributed DataSet, in which we use a RDD to represent the data sequence.

  18. def toLocal(): LocalDataSet[D]

    Convert current DataSet to a local DataSet, in which we use an iterator to represent the data sequence.

  19. def toString(): String
    Definition Classes
    AnyRef → Any
  20. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  21. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  22. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from AnyRef

Inherited from Any

Ungrouped