Package io.github.ggerganov.whispercpp

Interface WhisperCppJnaLibrary

All Superinterfaces:
com.sun.jna.Library
public interface WhisperCppJnaLibrary extends com.sun.jna.Library

  • Field Details

  • Method Details

    • whisper_print_system_info

      String whisper_print_system_info()

    • whisper_init_from_file

      com.sun.jna.Pointer whisper_init_from_file(String path_model)
      Allocate (almost) all memory needed for the model by loading from a file.
      Parameters:
      path_model - Path to the model file
      Returns:
      Whisper context on success, null on failure

    • whisper_init_from_buffer

      com.sun.jna.Pointer whisper_init_from_buffer(com.sun.jna.Pointer buffer, int buffer_size)
      Allocate (almost) all memory needed for the model by loading from a buffer.
      Parameters:
      buffer - Model buffer
      buffer_size - Size of the model buffer
      Returns:
      Whisper context on success, null on failure

    • whisper_init

      com.sun.jna.Pointer whisper_init(WhisperModelLoader loader)
      Allocate (almost) all memory needed for the model using a model loader.
      Parameters:
      loader - Model loader
      Returns:
      Whisper context on success, null on failure

    • whisper_init_from_file_no_state

      com.sun.jna.Pointer whisper_init_from_file_no_state(String path_model)
      Allocate (almost) all memory needed for the model by loading from a file without allocating the state.
      Parameters:
      path_model - Path to the model file
      Returns:
      Whisper context on success, null on failure

    • whisper_init_from_buffer_no_state

      com.sun.jna.Pointer whisper_init_from_buffer_no_state(com.sun.jna.Pointer buffer, int buffer_size)
      Allocate (almost) all memory needed for the model by loading from a buffer without allocating the state.
      Parameters:
      buffer - Model buffer
      buffer_size - Size of the model buffer
      Returns:
      Whisper context on success, null on failure

    • whisper_init_no_state

      com.sun.jna.Pointer whisper_init_no_state(WhisperModelLoader loader)
      Allocate (almost) all memory needed for the model using a model loader without allocating the state.
      Parameters:
      loader - Model loader
      Returns:
      Whisper context on success, null on failure

    • whisper_init_state

      com.sun.jna.Pointer whisper_init_state(com.sun.jna.Pointer ctx)
      Allocate memory for the Whisper state.
      Parameters:
      ctx - Whisper context
      Returns:
      Whisper state on success, null on failure

    • whisper_free

      void whisper_free(com.sun.jna.Pointer ctx)
      Free all allocated memory associated with the Whisper context.
      Parameters:
      ctx - Whisper context

    • whisper_free_state

      void whisper_free_state(com.sun.jna.Pointer state)
      Free all allocated memory associated with the Whisper state.
      Parameters:
      state - Whisper state

    • whisper_pcm_to_mel

      int whisper_pcm_to_mel(com.sun.jna.Pointer ctx, float[] samples, int n_samples, int n_threads)
      Convert RAW PCM audio to log mel spectrogram. The resulting spectrogram is stored inside the default state of the provided whisper context.
      Parameters:
      ctx - - Pointer to a WhisperContext
      Returns:
      0 on success

    • whisper_pcm_to_mel_with_state

      int whisper_pcm_to_mel_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, float[] samples, int n_samples, int n_threads)
      Parameters:
      ctx - Pointer to a WhisperContext
      state - Pointer to WhisperState
      n_samples -
      n_threads -
      Returns:
      0 on success

    • whisper_set_mel

      int whisper_set_mel(com.sun.jna.Pointer ctx, float[] data, int n_len, int n_mel)
      This can be used to set a custom log mel spectrogram inside the default state of the provided whisper context. Use this instead of whisper_pcm_to_mel() if you want to provide your own log mel spectrogram. n_mel must be 80
      Returns:
      0 on success

    • whisper_set_mel_with_state

      int whisper_set_mel_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, float[] data, int n_len, int n_mel)

    • whisper_encode

      int whisper_encode(com.sun.jna.Pointer ctx, int offset, int n_threads)
      Run the Whisper encoder on the log mel spectrogram stored inside the default state in the provided whisper context. Make sure to call whisper_pcm_to_mel() or whisper_set_mel() first. Offset can be used to specify the offset of the first frame in the spectrogram.
      Returns:
      0 on success

    • whisper_encode_with_state

      int whisper_encode_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, int offset, int n_threads)

    • whisper_decode

      int whisper_decode(com.sun.jna.Pointer ctx, com.sun.jna.Pointer tokens, int n_tokens, int n_past, int n_threads)
      Run the Whisper decoder to obtain the logits and probabilities for the next token. Make sure to call whisper_encode() first. tokens + n_tokens is the provided context for the decoder. n_past is the number of tokens to use from previous decoder calls. Returns 0 on success TODO: add support for multiple decoders

    • whisper_decode_with_state

      int whisper_decode_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, com.sun.jna.Pointer tokens, int n_tokens, int n_past, int n_threads)
      Parameters:
      ctx -
      state -
      tokens - Pointer to int tokens
      n_tokens -
      n_past -
      n_threads -
      Returns:

    • whisper_tokenize

      int whisper_tokenize(com.sun.jna.Pointer ctx, String text, com.sun.jna.Pointer tokens, int n_max_tokens)
      Convert the provided text into tokens. The tokens pointer must be large enough to hold the resulting tokens. Returns the number of tokens on success, no more than n_max_tokens Returns -1 on failure TODO: not sure if correct

    • whisper_lang_max_id

      int whisper_lang_max_id()
      Largest language id (i.e. number of available languages - 1)

    • whisper_lang_id

      int whisper_lang_id(String lang)
      Returns:
      the id of the specified language, returns -1 if not found. Examples: "de" -> 2 "german" -> 2

    • whisper_lang_str

      String whisper_lang_str(int id)
      Returns:
      the short string of the specified language id (e.g. 2 -> "de"), returns nullptr if not found

    • whisper_lang_auto_detect

      int whisper_lang_auto_detect(com.sun.jna.Pointer ctx, int offset_ms, int n_threads, float[] lang_probs)
      Use mel data at offset_ms to try and auto-detect the spoken language. Make sure to call whisper_pcm_to_mel() or whisper_set_mel() first Returns the top language id or negative on failure If not null, fills the lang_probs array with the probabilities of all languages The array must be whisper_lang_max_id() + 1 in size ref: https://github.com/openai/whisper/blob/main/whisper/decoding.py#L18-L69

    • whisper_lang_auto_detect_with_state

      int whisper_lang_auto_detect_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, int offset_ms, int n_threads, float[] lang_probs)

    • whisper_n_len

      int whisper_n_len(com.sun.jna.Pointer ctx)

    • whisper_n_len_from_state

      int whisper_n_len_from_state(com.sun.jna.Pointer state)

    • whisper_n_vocab

      int whisper_n_vocab(com.sun.jna.Pointer ctx)

    • whisper_n_text_ctx

      int whisper_n_text_ctx(com.sun.jna.Pointer ctx)

    • whisper_n_audio_ctx

      int whisper_n_audio_ctx(com.sun.jna.Pointer ctx)

    • whisper_is_multilingual

      int whisper_is_multilingual(com.sun.jna.Pointer ctx)

    • whisper_model_n_vocab

      int whisper_model_n_vocab(com.sun.jna.Pointer ctx)

    • whisper_model_n_audio_ctx

      int whisper_model_n_audio_ctx(com.sun.jna.Pointer ctx)

    • whisper_model_n_audio_state

      int whisper_model_n_audio_state(com.sun.jna.Pointer ctx)

    • whisper_model_n_audio_head

      int whisper_model_n_audio_head(com.sun.jna.Pointer ctx)

    • whisper_model_n_audio_layer

      int whisper_model_n_audio_layer(com.sun.jna.Pointer ctx)

    • whisper_model_n_text_ctx

      int whisper_model_n_text_ctx(com.sun.jna.Pointer ctx)

    • whisper_model_n_text_state

      int whisper_model_n_text_state(com.sun.jna.Pointer ctx)

    • whisper_model_n_text_head

      int whisper_model_n_text_head(com.sun.jna.Pointer ctx)

    • whisper_model_n_text_layer

      int whisper_model_n_text_layer(com.sun.jna.Pointer ctx)

    • whisper_model_n_mels

      int whisper_model_n_mels(com.sun.jna.Pointer ctx)

    • whisper_model_ftype

      int whisper_model_ftype(com.sun.jna.Pointer ctx)

    • whisper_model_type

      int whisper_model_type(com.sun.jna.Pointer ctx)

    • whisper_get_logits

      float[] whisper_get_logits(com.sun.jna.Pointer ctx)
      Token logits obtained from the last call to whisper_decode(). The logits for the last token are stored in the last row Rows: n_tokens Cols: n_vocab

    • whisper_get_logits_from_state

      float[] whisper_get_logits_from_state(com.sun.jna.Pointer state)

    • whisper_token_to_str

      String whisper_token_to_str(com.sun.jna.Pointer ctx, int token)

    • whisper_model_type_readable

      String whisper_model_type_readable(com.sun.jna.Pointer ctx)

    • whisper_token_eot

      int whisper_token_eot(com.sun.jna.Pointer ctx)

    • whisper_token_sot

      int whisper_token_sot(com.sun.jna.Pointer ctx)

    • whisper_token_prev

      int whisper_token_prev(com.sun.jna.Pointer ctx)

    • whisper_token_solm

      int whisper_token_solm(com.sun.jna.Pointer ctx)

    • whisper_token_not

      int whisper_token_not(com.sun.jna.Pointer ctx)

    • whisper_token_beg

      int whisper_token_beg(com.sun.jna.Pointer ctx)

    • whisper_token_lang

      int whisper_token_lang(com.sun.jna.Pointer ctx, int lang_id)

    • whisper_token_translate

      int whisper_token_translate(com.sun.jna.Pointer ctx)

    • whisper_token_transcribe

      int whisper_token_transcribe(com.sun.jna.Pointer ctx)

    • whisper_print_timings

      void whisper_print_timings(com.sun.jna.Pointer ctx)

    • whisper_reset_timings

      void whisper_reset_timings(com.sun.jna.Pointer ctx)

    • whisper_full_default_params_by_ref

      com.sun.jna.Pointer whisper_full_default_params_by_ref(int strategy)
      Provides default params which can be used with `whisper_full()` etc. Because this function allocates memory for the params, the caller must call either: - call `whisper_free_params()` - `Native.free(Pointer.nativeValue(pointer));`
      Parameters:
      strategy - - WhisperSamplingStrategy.value

    • whisper_free_params

      void whisper_free_params(com.sun.jna.Pointer params)

    • whisper_full

      int whisper_full(com.sun.jna.Pointer ctx, WhisperFullParams params, float[] samples, int n_samples)
      Run the entire model: PCM -> log mel spectrogram -> encoder -> decoder -> text Not thread safe for same context Uses the specified decoding strategy to obtain the text.

    • whisper_full_with_state

      int whisper_full_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, WhisperFullParams params, float[] samples, int n_samples)

    • whisper_full_parallel

      int whisper_full_parallel(com.sun.jna.Pointer ctx, WhisperFullParams params, float[] samples, int n_samples, int n_processors)

    • whisper_full_n_segments

      int whisper_full_n_segments(com.sun.jna.Pointer ctx)
      Number of generated text segments. A segment can be a few words, a sentence, or even a paragraph.
      Parameters:
      ctx - Pointer to WhisperContext

    • whisper_full_n_segments_from_state

      int whisper_full_n_segments_from_state(com.sun.jna.Pointer state)
      Parameters:
      state - Pointer to WhisperState

    • whisper_full_lang_id

      int whisper_full_lang_id(com.sun.jna.Pointer ctx)
      Language id associated with the context's default state.
      Parameters:
      ctx - Pointer to WhisperContext

    • whisper_full_lang_id_from_state

      int whisper_full_lang_id_from_state(com.sun.jna.Pointer state)
      Language id associated with the provided state

    • whisper_pcm_to_mel_phase_vocoder

      int whisper_pcm_to_mel_phase_vocoder(com.sun.jna.Pointer ctx, float[] samples, int n_samples, int n_threads)
      Convert RAW PCM audio to log mel spectrogram but applies a Phase Vocoder to speed up the audio x2. The resulting spectrogram is stored inside the default state of the provided whisper context.
      Returns:
      0 on success

    • whisper_pcm_to_mel_phase_vocoder_with_state

      int whisper_pcm_to_mel_phase_vocoder_with_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, float[] samples, int n_samples, int n_threads)

    • whisper_full_get_segment_t0

      long whisper_full_get_segment_t0(com.sun.jna.Pointer ctx, int i_segment)
      Get the start time of the specified segment.

    • whisper_full_get_segment_t0_from_state

      long whisper_full_get_segment_t0_from_state(com.sun.jna.Pointer state, int i_segment)
      Get the start time of the specified segment from the state.

    • whisper_full_get_segment_t1

      long whisper_full_get_segment_t1(com.sun.jna.Pointer ctx, int i_segment)
      Get the end time of the specified segment.

    • whisper_full_get_segment_t1_from_state

      long whisper_full_get_segment_t1_from_state(com.sun.jna.Pointer state, int i_segment)
      Get the end time of the specified segment from the state.

    • whisper_full_get_segment_text

      String whisper_full_get_segment_text(com.sun.jna.Pointer ctx, int i_segment)
      Get the text of the specified segment.

    • whisper_full_get_segment_text_from_state

      String whisper_full_get_segment_text_from_state(com.sun.jna.Pointer state, int i_segment)
      Get the text of the specified segment from the state.

    • whisper_full_n_tokens

      int whisper_full_n_tokens(com.sun.jna.Pointer ctx, int i_segment)
      Get the number of tokens in the specified segment.

    • whisper_full_n_tokens_from_state

      int whisper_full_n_tokens_from_state(com.sun.jna.Pointer state, int i_segment)
      Get the number of tokens in the specified segment from the state.

    • whisper_full_get_token_text

      String whisper_full_get_token_text(com.sun.jna.Pointer ctx, int i_segment, int i_token)
      Get the token text of the specified token in the specified segment.

    • whisper_full_get_token_text_from_state

      String whisper_full_get_token_text_from_state(com.sun.jna.Pointer ctx, com.sun.jna.Pointer state, int i_segment, int i_token)
      Get the token text of the specified token in the specified segment from the state.

    • whisper_full_get_token_id

      int whisper_full_get_token_id(com.sun.jna.Pointer ctx, int i_segment, int i_token)
      Get the token ID of the specified token in the specified segment.

    • whisper_full_get_token_id_from_state

      int whisper_full_get_token_id_from_state(com.sun.jna.Pointer state, int i_segment, int i_token)
      Get the token ID of the specified token in the specified segment from the state.

    • whisper_full_get_token_data

      WhisperTokenData whisper_full_get_token_data(com.sun.jna.Pointer ctx, int i_segment, int i_token)
      Get token data for the specified token in the specified segment.

    • whisper_full_get_token_data_from_state

      WhisperTokenData whisper_full_get_token_data_from_state(com.sun.jna.Pointer state, int i_segment, int i_token)
      Get token data for the specified token in the specified segment from the state.

    • whisper_full_get_token_p

      float whisper_full_get_token_p(com.sun.jna.Pointer ctx, int i_segment, int i_token)
      Get the probability of the specified token in the specified segment.

    • whisper_full_get_token_p_from_state

      float whisper_full_get_token_p_from_state(com.sun.jna.Pointer state, int i_segment, int i_token)
      Get the probability of the specified token in the specified segment from the state.

    • whisper_bench_memcpy

      int whisper_bench_memcpy(int nThreads)
      Benchmark function for memcpy.
      Parameters:
      nThreads - Number of threads to use for the benchmark.
      Returns:
      The result of the benchmark.

    • whisper_bench_memcpy_str

      String whisper_bench_memcpy_str(int nThreads)
      Benchmark function for memcpy as a string.
      Parameters:
      nThreads - Number of threads to use for the benchmark.
      Returns:
      The result of the benchmark as a string.

    • whisper_bench_ggml_mul_mat

      int whisper_bench_ggml_mul_mat(int nThreads)
      Benchmark function for ggml_mul_mat.
      Parameters:
      nThreads - Number of threads to use for the benchmark.
      Returns:
      The result of the benchmark.

    • whisper_bench_ggml_mul_mat_str

      String whisper_bench_ggml_mul_mat_str(int nThreads)
      Benchmark function for ggml_mul_mat as a string.
      Parameters:
      nThreads - Number of threads to use for the benchmark.
      Returns:
      The result of the benchmark as a string.