ai.djl.basicdataset.utils

Class TextData.Configuration

java.lang.Object

ai.djl.basicdataset.utils.TextData.Configuration

Enclosing class:

TextData

public static final class TextData.Configuration
extends java.lang.Object

The configuration for creating a TextData value in a Dataset.

Constructor Summary

Constructors

Constructor and Description
Configuration()

Method Summary

Modifier and Type	Method and Description
TextData.Configuration	setEmbeddingSize(int embeddingSize) Sets the size for new TextEmbeddings.
TextData.Configuration	setReservedTokens(java.util.List<java.lang.String> reservedTokens) Sets the list of reserved tokens.
TextData.Configuration	setTextEmbedding(ai.djl.modality.nlp.embedding.TextEmbedding textEmbedding) Sets the TextEmbedding to use to embed the text data.
TextData.Configuration	setTextProcessors(java.util.List<ai.djl.modality.nlp.preprocess.TextProcessor> textProcessors) Sets the TextProcessors to use for the text data.
TextData.Configuration	setUnknownToken(java.lang.String unknownToken) Sets the default unknown token.
TextData.Configuration	setVocabulary(ai.djl.modality.nlp.Vocabulary vocabulary) Sets the Vocabulary to use to hold the text data.
TextData.Configuration	update(TextData.Configuration other) Updates this TextData.Configuration with the non-null values from another configuration.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Configuration

public Configuration()

Method Detail

setTextProcessors

public TextData.Configuration setTextProcessors(java.util.List<ai.djl.modality.nlp.preprocess.TextProcessor> textProcessors)

Sets the TextProcessors to use for the text data.

Parameters:

textProcessors - the TextProcessors

Returns:

this configuration

setTextEmbedding

public TextData.Configuration setTextEmbedding(ai.djl.modality.nlp.embedding.TextEmbedding textEmbedding)

Sets the TextEmbedding to use to embed the text data.

Parameters:

textEmbedding - the TextEmbedding

Returns:

this configuration

setVocabulary

public TextData.Configuration setVocabulary(ai.djl.modality.nlp.Vocabulary vocabulary)

Sets the Vocabulary to use to hold the text data.

Parameters:

vocabulary - the Vocabulary

Returns:

this configuration

setEmbeddingSize

public TextData.Configuration setEmbeddingSize(int embeddingSize)

Sets the size for new TextEmbeddings.

Parameters:

embeddingSize - the embedding size

Returns:

this configuration

setUnknownToken

public TextData.Configuration setUnknownToken(java.lang.String unknownToken)

Sets the default unknown token.

Parameters:

unknownToken - the String value of unknown token

Returns:

this configuration

setReservedTokens

public TextData.Configuration setReservedTokens(java.util.List<java.lang.String> reservedTokens)

Sets the list of reserved tokens.

Parameters:

reservedTokens - true to train the text embedding

Returns:

this configuration

update

public TextData.Configuration update(TextData.Configuration other)

Updates this TextData.Configuration with the non-null values from another configuration.

Parameters:

other - the other configuration to use to update this

Returns:

this configuration after updating