ai.djl.basicdataset

Class TatoebaEnglishFrenchDataset

java.lang.Object

ai.djl.training.dataset.RandomAccessDataset

ai.djl.basicdataset.TatoebaEnglishFrenchDataset

All Implemented Interfaces:

ai.djl.repository.dataset.PreparedDataset, ai.djl.repository.dataset.ZooDataset, ai.djl.training.dataset.Dataset, java.util.RandomAccess

public class TatoebaEnglishFrenchDataset
extends ai.djl.training.dataset.RandomAccessDataset
implements ai.djl.repository.dataset.ZooDataset

TatoebaEnglishFrenchDataset is a English-French machine translation dataset from The Tatoeba Project (http://www.manythings.org/anki/).

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TatoebaEnglishFrenchDataset.Builder A builder for a TatoebaEnglishFrenchDataset.

Nested classes/interfaces inherited from class ai.djl.training.dataset.RandomAccessDataset

ai.djl.training.dataset.RandomAccessDataset.BaseBuilder<T extends ai.djl.training.dataset.RandomAccessDataset.BaseBuilder>

Nested classes/interfaces inherited from interface ai.djl.training.dataset.Dataset

ai.djl.training.dataset.Dataset.Usage

Field Summary

Fields inherited from class ai.djl.training.dataset.RandomAccessDataset

batchifier, device, executor, limit, pipeline, prefetchNumber, sampler, targetPipeline

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	TatoebaEnglishFrenchDataset(TatoebaEnglishFrenchDataset.Builder builder) Creates a new instance of TatoebaEnglishFrenchDataset.

Method Summary

Modifier and Type	Method and Description
protected long	availableSize()
static TatoebaEnglishFrenchDataset.Builder	builder() Creates a new builder to build a TatoebaEnglishFrenchDataset.
ai.djl.training.dataset.Record	get(ai.djl.ndarray.NDManager manager, long index)
ai.djl.repository.Artifact	getArtifact()
ai.djl.repository.MRL	getMrl()
ai.djl.repository.Repository	getRepository()
ai.djl.training.dataset.Dataset.Usage	getUsage()
boolean	isPrepared()
void	prepareData(ai.djl.training.dataset.Dataset.Usage usage)
void	setPrepared(boolean prepared)
void	useDefaultArtifact()

Methods inherited from class ai.djl.training.dataset.RandomAccessDataset

getData, getData, randomSplit, size

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface ai.djl.repository.dataset.ZooDataset

prepare

Methods inherited from interface ai.djl.training.dataset.Dataset

getData

Methods inherited from interface ai.djl.repository.dataset.PreparedDataset

prepare

Constructor Detail

TatoebaEnglishFrenchDataset

protected TatoebaEnglishFrenchDataset(TatoebaEnglishFrenchDataset.Builder builder)

Creates a new instance of TatoebaEnglishFrenchDataset.

Parameters:

builder - the builder object to build from

Method Detail

builder

public static TatoebaEnglishFrenchDataset.Builder builder()

Creates a new builder to build a TatoebaEnglishFrenchDataset.

Returns:

a new builder

getMrl

public ai.djl.repository.MRL getMrl()

Specified by:

getMrl in interface ai.djl.repository.dataset.ZooDataset

getRepository

public ai.djl.repository.Repository getRepository()

Specified by:

getRepository in interface ai.djl.repository.dataset.ZooDataset

getArtifact

public ai.djl.repository.Artifact getArtifact()

Specified by:

getArtifact in interface ai.djl.repository.dataset.ZooDataset

getUsage

public ai.djl.training.dataset.Dataset.Usage getUsage()

Specified by:

getUsage in interface ai.djl.repository.dataset.ZooDataset

isPrepared

public boolean isPrepared()

Specified by:

isPrepared in interface ai.djl.repository.dataset.ZooDataset

setPrepared

public void setPrepared(boolean prepared)

Specified by:

setPrepared in interface ai.djl.repository.dataset.ZooDataset

useDefaultArtifact

public void useDefaultArtifact()
                        throws java.io.IOException

Specified by:

useDefaultArtifact in interface ai.djl.repository.dataset.ZooDataset

Throws:

java.io.IOException

prepareData

public void prepareData(ai.djl.training.dataset.Dataset.Usage usage)
                 throws java.io.IOException

Specified by:

prepareData in interface ai.djl.repository.dataset.ZooDataset

Throws:

java.io.IOException

get

public ai.djl.training.dataset.Record get(ai.djl.ndarray.NDManager manager,
                                          long index)
                                   throws ai.djl.modality.nlp.EmbeddingException

Specified by:

get in class ai.djl.training.dataset.RandomAccessDataset

Throws:

ai.djl.modality.nlp.EmbeddingException

availableSize

protected long availableSize()

Specified by:

availableSize in class ai.djl.training.dataset.RandomAccessDataset