Package ai.picovoice.picollm

Class PicoLLMGenerateParams

  • public class PicoLLMGenerateParams
extends java.lang.Object
    Class for configuring generation parameters when calling `PicoLLM.generate()`.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      static class  PicoLLMGenerateParams.Builder
      Builder class for creating a `PicoLLMGenerateParams` instance.

    • Constructor Summary

      Constructors 
      Constructor Description
      PicoLLMGenerateParams​(int completionTokenLimit, java.lang.String[] stopPhrases, int seed, float presencePenalty, float frequencyPenalty, float temperature, float topP, int numTopChoices, PicoLLMStreamCallback streamCallback)
      Constructor.

    • Constructor Detail

      • PicoLLMGenerateParams

        public PicoLLMGenerateParams​(int completionTokenLimit,
                             java.lang.String[] stopPhrases,
                             int seed,
                             float presencePenalty,
                             float frequencyPenalty,
                             float temperature,
                             float topP,
                             int numTopChoices,
                             PicoLLMStreamCallback streamCallback)
        Constructor.
        Parameters:
        completionTokenLimit - The maximum number of tokens allowed in the completion. If the generation process stops due to reaching this limit, the endpoint output argument will indicate a completion token limit reached condition. Set to `-1` to impose no limit.
        stopPhrases - Phrases that trigger early completion termination. The generation process stops when it encounters any of these phrases in the completion. The already generated completion, including the encountered stop phrase, will be returned. Set to `null` to turn off this feature.
        seed - The seed value for the internal random number generator. Seeding enforces deterministic outputs. Set to -1 for randomized outputs for a given prompt.
        presencePenalty - Penalty for presence of tokens in the generated completion. It penalizes logits already appearing in the partial completion if set to a positive value. If set to `0.0`, it has no effect.
        frequencyPenalty - Penalty for the frequency of tokens in the generated completion. If set to a positive floating-point value, it penalizes logits proportional to the frequency of their appearance in the partial completion. If set to `0.0`, it has no effect.
        temperature - Sampling temperature. A higher temperature smoothens the sampler's output, increasing the randomness. In contrast, a lower temperature creates a narrower distribution and reduces variability. Setting it to `0` selects the maximum logit during sampling.
        topP - A positive floating-point number within (0, 1]. It restricts the sampler's choices to high-probability logits that form the `topP` portion of the probability mass. Hence, it avoids randomly selecting unlikely logits. A value of `1.0` enables the sampler to pick any token with non-zero probability, turning off the feature.
        numTopChoices - If set to a positive value, picoLLM returns the list of the highest probability tokens for any generated token. Set to `0` to turn off the feature. The maximum number of top choices is determined by `PicoLLM.getMaxTopChoices()`.
        streamCallback - If not set to `null`, picoLLM executes this callback every time a new piece of completion string becomes available.

    • Method Detail

      • getCompletionTokenLimit

        public java.lang.Integer getCompletionTokenLimit()
        Gets the maximum number of tokens allowed in the completion.
        Returns:
        The completion token limit.

      • setCompletionTokenLimit

        public void setCompletionTokenLimit​(java.lang.Integer completionTokenLimit)
        Sets the maximum number of tokens allowed in the completion.
        Parameters:
        completionTokenLimit - The completion token limit to set. If the generation process stops due to reaching this limit, the endpoint output argument will indicate a completion token limit reached condition. Set to `null` to impose no limit.

      • getStopPhrases

        public java.lang.String[] getStopPhrases()
        Gets the phrases that trigger early completion termination.
        Returns:
        Phrases that trigger early completion termination.

      • setStopPhrases

        public void setStopPhrases​(java.lang.String[] stopPhrases)
        Sets the phrases that trigger early completion termination.
        Parameters:
        stopPhrases - Phrases that trigger early completion termination. The generation process stops when it encounters any of these phrases in the completion. The already generated completion, including the encountered stop phrase, will be returned. Set to `null` to turn off this feature.

      • getSeed

        public java.lang.Integer getSeed()
        Gets the seed value for the internal random number generator.
        Returns:
        The seed value for the internal random number generator. Seeding enforces deterministic outputs. Set to `null` for randomized outputs for a given prompt.

      • setSeed

        public void setSeed​(java.lang.Integer seed)
        Sets the seed value for the internal random number generator.
        Parameters:
        seed - The seed value for the internal random number generator. Seeding enforces deterministic outputs. Set to `-1` for randomized outputs for a given prompt.

      • getPresencePenalty

        public float getPresencePenalty()
        Gets the presence penalty.
        Returns:
        The presence penalty. It penalizes logits already appearing in the partial completion if set to a positive value. If set to `0.0`, it has no effect.

      • setPresencePenalty

        public void setPresencePenalty​(float presencePenalty)
        Sets the presence penalty.
        Parameters:
        presencePenalty - The presence penalty. It penalizes logits already appearing in the partial completion if set to a positive value. If set to `0.0`, it has no effect.

      • getFrequencyPenalty

        public float getFrequencyPenalty()
        Gets the frequency penalty.
        Returns:
        The frequency penalty. If set to a positive floating-point value, it penalizes logits proportional to the frequency of their appearance in the partial completion. If set to `0.0`, it has no effect.

      • setFrequencyPenalty

        public void setFrequencyPenalty​(float frequencyPenalty)
        Sets the frequency penalty.
        Parameters:
        frequencyPenalty - The frequency penalty. If set to a positive floating-point value, it penalizes logits proportional to the frequency of their appearance in the partial completion. If set to `0.0`, it has no effect.

      • getTemperature

        public float getTemperature()
        Gets the temperature parameter.
        Returns:
        The temperature parameter. A higher temperature smoothens the sampler's output, increasing the randomness. In contrast, a lower temperature creates a narrower distribution and reduces variability. Setting it to `0` selects the maximum logit during sampling.

      • setTemperature

        public void setTemperature​(float temperature)
        Sets the temperature parameter.
        Parameters:
        temperature - The temperature parameter. A higher temperature smoothens the sampler's output, increasing the randomness. In contrast, a lower temperature creates a narrower distribution and reduces variability. Setting it to `0` selects the maximum logit during sampling.

      • getTopP

        public float getTopP()
        Gets the top-p parameter.
        Returns:
        The top-p parameter. A positive floating-point number within (0, 1]. It restricts the sampler's choices to high-probability logits that form the top_p portion of the probability mass. Hence, it avoids randomly selecting unlikely logits. A value of 1.0 enables the sampler to pick any token with non-zero probability, turning off the feature.

      • setTopP

        public void setTopP​(float topP)
        Sets the top-p parameter.
        Parameters:
        topP - The top-p parameter. A positive floating-point number within (0, 1]. It restricts the sampler's choices to high-probability logits that form the top_p portion of the probability mass. Hence, it avoids randomly selecting unlikely logits. A value of `1.0` enables the sampler to pick any token with non-zero probability, turning off the feature.

      • getNumTopChoices

        public int getNumTopChoices()
        Gets the number of top choices.
        Returns:
        The number of top choices. If set to a positive value, picoLLM returns the list of the highest probability tokens for any generated token. Set to `0` to turn off the feature. The maximum number of top choices is determined by `PicoLLM.getMaxTopChoices()`.

      • setNumTopChoices

        public void setNumTopChoices​(int numTopChoices)
        Sets the number of top choices.
        Parameters:
        numTopChoices - The number of top choices. If set to a positive value, picoLLM returns the list of the highest probability tokens for any generated token. Set to 0 to turn off the feature. The maximum number of top choices is determined by `PicoLLM.getMaxTopChoices()`.

      • getStreamCallback

        public PicoLLMStreamCallback getStreamCallback()
        Gets the completion stream callback.
        Returns:
        The stream callback. If not set to null, picoLLM executes this callback every time a new piece of completion string becomes available.

      • setStreamCallback

        public void setStreamCallback​(PicoLLMStreamCallback streamCallback)
        Sets the completion stream callback.
        Parameters:
        streamCallback - The stream callback. If not set to null, picoLLM executes this callback every time a new piece of completion string becomes available.