Package ai.onnxruntime

package ai.onnxruntime
A Java interface to the ONNX Runtime.

Provides access to the same execution backends as the C library. Non-representable types in Java (such as fp16) are converted into the nearest Java primitive type when accessed through this API.

There are two shared libraries required: onnxruntime and onnxruntime4j_jni . The loader is in OnnxRuntime and the logic is in this order:

  1. The user may signal to skip loading of a shared library using a property in the form onnxruntime.native.LIB_NAME.skip with a value of true. This means the user has decided to load the library by some other means.
  2. The user may specify an explicit location of all native library files using a property in the form onnxruntime.native.path. This uses System.load(java.lang.String).
  3. The user may specify an explicit location of the shared library file using a property in the form onnxruntime.native.LIB_NAME.path. This uses System.load(java.lang.String).
  4. The shared library is autodiscovered:
    1. If the shared library is present in the classpath resources, load using System.load(java.lang.String) via a temporary file. Ideally, this should be the default use case when adding JAR's/dependencies containing the shared libraries to your classpath.
    2. If the shared library is not present in the classpath resources, then load using System.loadLibrary(java.lang.String), which usually looks elsewhere on the filesystem for the library. The semantics and behavior of that method are system/JVM dependent. Typically, the java.library.path property is used to specify the location of native libraries.
For troubleshooting, all shared library loading events are reported to Java logging at the level FINE.

Note that CUDA, ROCM, DNNL, OpenVINO and TensorRT are all "shared library execution providers" and must be stored either in the directory containing the ONNX Runtime core native library, or as a classpath resource. This is because these providers are loaded by the ONNX Runtime native library itself and the Java API cannot control the loading location.