Package ai.picovoice.picollm

Class PicoLLM

java.lang.Object

ai.picovoice.picollm.PicoLLM

public class PicoLLM
extends Object

Android binding for picoLLM.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	PicoLLM.Builder	Builder class for creating a `PicoLLM` instance.
class	PicoLLM.DialogBuilder	Builder class for creating a `PicoLLMDialog` instance.

Method Summary

Modifier and Type	Method	Description
void	delete()	Deletes the resources acquired by picoLLM.
float[]	forward(int token)	Performs a single forward pass given a token and returns the logits.
PicoLLMCompletion	generate(String prompt, PicoLLMGenerateParams generateParams)	Generates completion text and relevant metadata given a text prompt and a set of generation parameters.
static String[]	getAvailableDevices()	Lists all available devices that picoLLM can use for inference.
int	getContextLength()	Getter for model's context length.
PicoLLM.DialogBuilder	getDialogBuilder()	Retrieves a new instance of DialogBuilder for constructing dialog objects from the currently loaded picoLLM model.
static int	getMaxTopChoices()	Retrieves the maximum number of top choices allowed during generation.
String	getModel()	Getter for model's name.
static String	getVersion()	Retrieves the version of the picoLLM library.
void	interrupt()	Interrupts `.generate()` if generation is in progress.
void	reset()	Resets the internal state of LLM.
static void	setSdk(String sdk)
int[]	tokenize(String text, boolean bos, boolean eos)	Tokenizes a given text using the model's tokenizer.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Details

setSdk

public static void setSdk(String sdk)

getVersion

public static String getVersion()
                         throws PicoLLMException

Retrieves the version of the picoLLM library.

Returns:

Version of the picoLLM library.

Throws:

PicoLLMException - if getting the version fails.

getMaxTopChoices

public static int getMaxTopChoices()

Retrieves the maximum number of top choices allowed during generation.

Returns:

Maximum number of top choices allowed during generation.

getAvailableDevices

public static String[] getAvailableDevices()
                                    throws PicoLLMException

Lists all available devices that picoLLM can use for inference. Each entry in the list can be the `device` when initializing picoLLM.

Returns:

Array of all available devices that picoLLM can be used for inference.

Throws:

PicoLLMException - if getting available devices fails.

delete

public void delete()

Deletes the resources acquired by picoLLM.

generate

public PicoLLMCompletion generate(String prompt,
 PicoLLMGenerateParams generateParams)
                           throws PicoLLMException

Generates completion text and relevant metadata given a text prompt and a set of generation parameters.

Parameters:

prompt - Text prompt.

generateParams - Generation parameters.

Returns:

Completion result.

Throws:

PicoLLMException - if generation fails.

interrupt

public void interrupt()
               throws PicoLLMException

Interrupts `.generate()` if generation is in progress. Otherwise, it has no effect.

Throws:

PicoLLMException - if interrupt fails.

tokenize

public int[] tokenize(String text,
 boolean bos,
 boolean eos)
               throws PicoLLMException

Tokenizes a given text using the model's tokenizer. This is a low-level function meant for benchmarking and advanced usage. `.generate()` should be used when possible.

Parameters:

text - Text.

bos - If set to `true`, the tokenizer prepends the beginning of the sentence token to the result.

eos - If set to `true`, the tokenizer appends the end of the sentence token to the result.

Returns:

Tokens representing the input text.

Throws:

PicoLLMException - if tokenization fails.

forward

public float[] forward(int token)
                throws PicoLLMException

Performs a single forward pass given a token and returns the logits. This is a low-level function for benchmarking and advanced usage. `.generate()` should be used when possible.

Parameters:

token - Input token.

Returns:

Logits.

Throws:

PicoLLMException - if the forward pass fails.

reset

public void reset()
           throws PicoLLMException

Resets the internal state of LLM. It should be called in conjunction with `.forward()` when processing a new sequence of tokens. This is a low-level function for benchmarking and advanced usage. `.generate()` should be used when possible.

Throws:

PicoLLMException - if resetting fails.

getModel

public String getModel()

Getter for model's name.

Returns:

Model's name.

getContextLength

public int getContextLength()

Getter for model's context length.

Returns:

Model's context length.

getDialogBuilder

public PicoLLM.DialogBuilder getDialogBuilder()

Retrieves a new instance of DialogBuilder for constructing dialog objects from the currently loaded picoLLM model.

Returns:

A new instance of DialogBuilder.