Package com.knuddels.jtokkit.api

Class EncodingResult

java.lang.Object

com.knuddels.jtokkit.api.EncodingResult

public final class EncodingResult
extends Object

The result of encoding operation.

Constructor Summary

Constructors

Constructor	Description
EncodingResult(IntArrayList tokens, boolean truncated)
EncodingResult(IntArrayList tokens, boolean truncated, int lastProcessedCharacterIndex)

Method Summary

Modifier and Type	Method	Description
int	getLastProcessedCharacterIndex()	Returns the index of the last processed character in the input string
IntArrayList	getTokens()	Returns the list of token ids
boolean	isTruncated()	Returns true if the token list was truncated because the maximum token length was exceeded
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Details

EncodingResult

public EncodingResult(IntArrayList tokens,
 boolean truncated)

EncodingResult

public EncodingResult(IntArrayList tokens,
 boolean truncated,
 int lastProcessedCharacterIndex)

Method Details

getTokens

public IntArrayList getTokens()

Returns the list of token ids

Returns:

the list of token ids

isTruncated

public boolean isTruncated()

Returns true if the token list was truncated because the maximum token length was exceeded

Returns:

true if the token list was truncated because the maximum token length was exceeded

getLastProcessedCharacterIndex

public int getLastProcessedCharacterIndex()

Returns the index of the last processed character in the input string

Returns:

the index of the last processed character in the input string, is -1 if text was null or empty

toString

public String toString()

Overrides:

toString in class Object