Package ai.djl.basicdataset.tabular

Class CsvDataset

  • All Implemented Interfaces:
    ai.djl.training.dataset.Dataset
    Direct Known Subclasses:
    AirfoilRandomAccess, AmazonReview, AmesRandomAccess
    public class CsvDataset
extends ai.djl.training.dataset.RandomAccessDataset
    CsvDataset represents the dataset that stored in a .csv file.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  CsvDataset.CsvBuilder<T extends CsvDataset.CsvBuilder<T>>
      Used to build a CsvDataset.
      static class  CsvDataset.Feature
      A class contains feature name and its Featurizer.
      static interface  CsvDataset.Featurizer
      An interface that convert String to numeric data.

      • Nested classes/interfaces inherited from class ai.djl.training.dataset.RandomAccessDataset

        ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T extends ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T>>

      • Nested classes/interfaces inherited from interface ai.djl.training.dataset.Dataset

        ai.djl.training.dataset.Dataset.Usage

    • Field Summary

      Fields 
      Modifier and Type Field Description
      protected org.apache.commons.csv.CSVFormat csvFormat  
      protected java.util.List<org.apache.commons.csv.CSVRecord> csvRecords  
      protected java.net.URL csvUrl  
      protected java.util.List<CsvDataset.Feature> features  
      protected java.util.List<CsvDataset.Feature> labels  

      • Fields inherited from class ai.djl.training.dataset.RandomAccessDataset

        dataBatchifier, device, labelBatchifier, limit, pipeline, prefetchNumber, sampler, targetPipeline

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      protected long availableSize()
      static CsvDataset.CsvBuilder<?> builder()
      Creates a builder to build a AmesRandomAccess.
      ai.djl.training.dataset.Record get​(ai.djl.ndarray.NDManager manager, long index)
      java.util.List<java.lang.String> getColumnNames()
      Returns the column names of the CSV file.
      void prepare​(ai.djl.util.Progress progress)
      protected ai.djl.ndarray.NDList toNDList​(ai.djl.ndarray.NDManager manager, org.apache.commons.csv.CSVRecord record, java.util.List<CsvDataset.Feature> selected)  

      • Methods inherited from class ai.djl.training.dataset.RandomAccessDataset

        getData, getData, getData, getData, randomSplit, size, subDataset, toArray

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

      • Methods inherited from interface ai.djl.training.dataset.Dataset

        prepare

    • Field Detail

      • csvUrl

        protected java.net.URL csvUrl

      • csvFormat

        protected org.apache.commons.csv.CSVFormat csvFormat

      • csvRecords

        protected java.util.List<org.apache.commons.csv.CSVRecord> csvRecords

    • Method Detail

      • get

        public ai.djl.training.dataset.Record get​(ai.djl.ndarray.NDManager manager,
                                          long index)
        Specified by:
        get in class ai.djl.training.dataset.RandomAccessDataset

      • availableSize

        protected long availableSize()
        Specified by:
        availableSize in class ai.djl.training.dataset.RandomAccessDataset

      • prepare

        public void prepare​(ai.djl.util.Progress progress)
             throws java.io.IOException
        Throws:
        java.io.IOException

      • getColumnNames

        public java.util.List<java.lang.String> getColumnNames()
        Returns the column names of the CSV file.
        Returns:
        a list of column name

      • toNDList

        protected ai.djl.ndarray.NDList toNDList​(ai.djl.ndarray.NDManager manager,
                                         org.apache.commons.csv.CSVRecord record,
                                         java.util.List<CsvDataset.Feature> selected)