Package ai.picovoice.picollm

Class PicoLLM

  • public class PicoLLM
extends java.lang.Object
    Android binding for picoLLM.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  PicoLLM.Builder
      Builder class for creating a `PicoLLM` instance.
      class  PicoLLM.DialogBuilder
      Builder class for creating a `PicoLLMDialog` instance.

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void delete()
      Deletes the resources acquired by picoLLM.
      float[] forward​(int token)
      Performs a single forward pass given a token and returns the logits.
      PicoLLMCompletion generate​(java.lang.String prompt, PicoLLMGenerateParams generateParams)
      Generates completion text and relevant metadata given a text prompt and a set of generation parameters.
      static java.lang.String[] getAvailableDevices()
      Lists all available devices that picoLLM can use for inference.
      int getContextLength()
      Getter for model's context length.
      PicoLLM.DialogBuilder getDialogBuilder()
      Retrieves a new instance of DialogBuilder for constructing dialog objects from the currently loaded picoLLM model.
      static int getMaxTopChoices()
      Retrieves the maximum number of top choices allowed during generation.
      java.lang.String getModel()
      Getter for model's name.
      static java.lang.String getVersion()
      Retrieves the version of the picoLLM library.
      void interrupt()
      Interrupts `.generate()` if generation is in progress.
      void reset()
      Resets the internal state of LLM.
      static void setSdk​(java.lang.String sdk)  
      int[] tokenize​(java.lang.String text, boolean bos, boolean eos)
      Tokenizes a given text using the model's tokenizer.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Method Detail

      • setSdk

        public static void setSdk​(java.lang.String sdk)

      • getVersion

        public static java.lang.String getVersion()
                                   throws PicoLLMException
        Retrieves the version of the picoLLM library.
        Returns:
        Version of the picoLLM library.
        Throws:
        PicoLLMException - if getting the version fails.

      • getMaxTopChoices

        public static int getMaxTopChoices()
        Retrieves the maximum number of top choices allowed during generation.
        Returns:
        Maximum number of top choices allowed during generation.

      • getAvailableDevices

        public static java.lang.String[] getAvailableDevices()
                                              throws PicoLLMException
        Lists all available devices that picoLLM can use for inference. Each entry in the list can be the `device` when initializing picoLLM.
        Returns:
        Array of all available devices that picoLLM can be used for inference.
        Throws:
        PicoLLMException - if getting available devices fails.

      • delete

        public void delete()
        Deletes the resources acquired by picoLLM.

      • generate

        public PicoLLMCompletion generate​(java.lang.String prompt,
                                  PicoLLMGenerateParams generateParams)
                           throws PicoLLMException
        Generates completion text and relevant metadata given a text prompt and a set of generation parameters.
        Parameters:
        prompt - Text prompt.
        generateParams - Generation parameters.
        Returns:
        Completion result.
        Throws:
        PicoLLMException - if generation fails.

      • interrupt

        public void interrupt()
               throws PicoLLMException
        Interrupts `.generate()` if generation is in progress. Otherwise, it has no effect.
        Throws:
        PicoLLMException - if interrupt fails.

      • tokenize

        public int[] tokenize​(java.lang.String text,
                      boolean bos,
                      boolean eos)
               throws PicoLLMException
        Tokenizes a given text using the model's tokenizer. This is a low-level function meant for benchmarking and advanced usage. `.generate()` should be used when possible.
        Parameters:
        text - Text.
        bos - If set to `true`, the tokenizer prepends the beginning of the sentence token to the result.
        eos - If set to `true`, the tokenizer appends the end of the sentence token to the result.
        Returns:
        Tokens representing the input text.
        Throws:
        PicoLLMException - if tokenization fails.

      • forward

        public float[] forward​(int token)
                throws PicoLLMException
        Performs a single forward pass given a token and returns the logits. This is a low-level function for benchmarking and advanced usage. `.generate()` should be used when possible.
        Parameters:
        token - Input token.
        Returns:
        Logits.
        Throws:
        PicoLLMException - if the forward pass fails.

      • reset

        public void reset()
           throws PicoLLMException
        Resets the internal state of LLM. It should be called in conjunction with `.forward()` when processing a new sequence of tokens. This is a low-level function for benchmarking and advanced usage. `.generate()` should be used when possible.
        Throws:
        PicoLLMException - if resetting fails.

      • getModel

        public java.lang.String getModel()
        Getter for model's name.
        Returns:
        Model's name.

      • getContextLength

        public int getContextLength()
        Getter for model's context length.
        Returns:
        Model's context length.

      • getDialogBuilder

        public PicoLLM.DialogBuilder getDialogBuilder()
        Retrieves a new instance of DialogBuilder for constructing dialog objects from the currently loaded picoLLM model.
        Returns:
        A new instance of DialogBuilder.