Package ai.onnxruntime

Class OnnxTensor

java.lang.Object

ai.onnxruntime.OnnxTensorLike

ai.onnxruntime.OnnxTensor

All Implemented Interfaces:

OnnxValue, AutoCloseable

public class OnnxTensor
extends OnnxTensorLike

A Java object wrapping an OnnxTensor. Tensors are the main input to the library, and can also be returned as outputs.

Nested Class Summary

Nested classes/interfaces inherited from interface ai.onnxruntime.OnnxValue

OnnxValue.OnnxValueType

Field Summary

Fields inherited from class ai.onnxruntime.OnnxTensorLike

allocatorHandle, closed, info, nativeHandle

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer).
static OnnxTensor	createTensor(OrtEnvironment env, Object data)	Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array.
static OnnxTensor	createTensor(OrtEnvironment env, String[] data, long[] shape)	Create a tensor from a flattened string array.
static OnnxTensor	createTensor(OrtEnvironment env, ByteBuffer data, long[] shape)	Create an OnnxTensor backed by a direct ByteBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, ByteBuffer data, long[] shape, OnnxJavaType type)	Create an OnnxTensor backed by a direct ByteBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, DoubleBuffer data, long[] shape)	Create an OnnxTensor backed by a direct DoubleBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, FloatBuffer data, long[] shape)	Create an OnnxTensor backed by a direct FloatBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, IntBuffer data, long[] shape)	Create an OnnxTensor backed by a direct IntBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, LongBuffer data, long[] shape)	Create an OnnxTensor backed by a direct LongBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, ShortBuffer data, long[] shape)	Create an OnnxTensor backed by a direct ShortBuffer.
static OnnxTensor	createTensor(OrtEnvironment env, ShortBuffer data, long[] shape, OnnxJavaType type)	Create an OnnxTensor backed by a direct ShortBuffer.
Optional<Buffer>	getBufferRef()	Returns a reference to the buffer which backs this OnnxTensor.
ByteBuffer	getByteBuffer()	Returns a copy of the underlying OnnxTensor as a ByteBuffer.
DoubleBuffer	getDoubleBuffer()	Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.
FloatBuffer	getFloatBuffer()	Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e.
IntBuffer	getIntBuffer()	Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.
LongBuffer	getLongBuffer()	Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.
ShortBuffer	getShortBuffer()	Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.
OnnxValue.OnnxValueType	getType()	Gets the type of this OnnxValue.
Object	getValue()	Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.
boolean	ownsBuffer()	Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.
String	toString()

Methods inherited from class ai.onnxruntime.OnnxTensorLike

checkClosed, getInfo, isClosed

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

ownsBuffer

public boolean ownsBuffer()

Returns true if the buffer in this OnnxTensor was created on construction of this tensor, i.e., it is a copy of a user supplied buffer or array and may hold the only reference to that buffer.

When this is true the backing buffer was copied from the user input, so users cannot mutate the state of this buffer without first getting the reference via getBufferRef().

Returns:

True if the buffer in this OnnxTensor was allocated by it on construction (i.e., it is a copy of a user buffer or array.)

getBufferRef

public Optional<Buffer> getBufferRef()

Returns a reference to the buffer which backs this OnnxTensor. If the tensor is not backed by a buffer (i.e., it is backed by memory allocated by ORT) this method returns an empty Optional.

Changes to the buffer elements will be reflected in the native OrtValue, this can be used to repeatedly update a single tensor for multiple different inferences without allocating new tensors, though the inputs must remain the same size and shape.

Note: the tensor could refer to a contiguous range of elements in this buffer, not the whole buffer. It is up to the user to manage this information by respecting the position and limit. As a consequence, accessing this reference should be considered problematic when multiple threads hold references to the buffer.

Returns:

A reference to the buffer.

getType

public OnnxValue.OnnxValueType getType()

Description copied from interface: OnnxValue

Gets the type of this OnnxValue.

Returns:

The value type.

getValue

public Object getValue()
                throws OrtException

Either returns a boxed primitive if the Tensor is a scalar, or a multidimensional array of primitives if it has multiple dimensions.

Java multidimensional arrays are quite slow for more than 2 dimensions, in that case it is recommended you use the Buffer extractors below (e.g., getFloatBuffer()).

Returns:

A Java value.

Throws:

OrtException - If the value could not be extracted as the Tensor is invalid, or if the native code encountered an error.

toString

public String toString()

Overrides:

toString in class Object

close

public void close()

Closes the tensor, releasing its underlying memory (if it's not backed by an NIO buffer). If it is backed by a buffer then the memory is released when the buffer is GC'd.

getByteBuffer

public ByteBuffer getByteBuffer()

Returns a copy of the underlying OnnxTensor as a ByteBuffer.

This method returns null if the OnnxTensor contains Strings as they are stored externally to the OnnxTensor.

Returns:

A ByteBuffer copy of the OnnxTensor.

getFloatBuffer

public FloatBuffer getFloatBuffer()

Returns a copy of the underlying OnnxTensor as a FloatBuffer if it can be losslessly converted into a float (i.e. it's a float, fp16 or bf16), otherwise it returns null.

Returns:

A FloatBuffer copy of the OnnxTensor.

getDoubleBuffer

public DoubleBuffer getDoubleBuffer()

Returns a copy of the underlying OnnxTensor as a DoubleBuffer if the underlying type is a double, otherwise it returns null.

Returns:

A DoubleBuffer copy of the OnnxTensor.

getShortBuffer

public ShortBuffer getShortBuffer()

Returns a copy of the underlying OnnxTensor as a ShortBuffer if the underlying type is int16, uint16, fp16 or bf16, otherwise it returns null.

Returns:

A ShortBuffer copy of the OnnxTensor.

getIntBuffer

public IntBuffer getIntBuffer()

Returns a copy of the underlying OnnxTensor as an IntBuffer if the underlying type is int32 or uint32, otherwise it returns null.

Returns:

An IntBuffer copy of the OnnxTensor.

getLongBuffer

public LongBuffer getLongBuffer()

Returns a copy of the underlying OnnxTensor as a LongBuffer if the underlying type is int64 or uint64, otherwise it returns null.

Returns:

A LongBuffer copy of the OnnxTensor.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 Object data)
                               throws OrtException

Create a Tensor from a Java primitive, primitive multidimensional array or String multidimensional array. The shape is inferred from the object using reflection. The default allocator is used.

Note: Java multidimensional arrays are not dense and this method requires traversing a large number of pointers for high dimensional arrays. For types other than Strings it is recommended to use one of the createTensor methods which accepts a Buffer, e.g. createTensor(OrtEnvironment, FloatBuffer, long[]) as those methods are zero copy to transfer data into ORT when using direct buffers.

Parameters:

env - The current OrtEnvironment.

data - The data to store in a tensor.

Returns:

An OnnxTensor storing the data.

Throws:

OrtException - If the onnx runtime threw an error.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 String[] data,
 long[] shape)
                               throws OrtException

Create a tensor from a flattened string array.

Requires the array to be flattened in row-major order. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data

shape - the shape of the tensor

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 FloatBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct FloatBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 DoubleBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct DoubleBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 ByteBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's OnnxJavaType.INT8.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 ByteBuffer data,
 long[] shape,
 OnnxJavaType type)
                               throws OrtException

Create an OnnxTensor backed by a direct ByteBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator. Tells the runtime it's the specified type.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

type - The type to use for the byte buffer elements.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 ShortBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct ShortBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 ShortBuffer data,
 long[] shape,
 OnnxJavaType type)
                               throws OrtException

Create an OnnxTensor backed by a direct ShortBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

type - The type of the data in the buffer, can be either OnnxJavaType.INT16, OnnxJavaType.FLOAT16 or OnnxJavaType.BFLOAT16.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 IntBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct IntBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.

createTensor

public static OnnxTensor createTensor(OrtEnvironment env,
 LongBuffer data,
 long[] shape)
                               throws OrtException

Create an OnnxTensor backed by a direct LongBuffer. The buffer should be in nativeOrder.

If the supplied buffer is not a direct buffer, a direct copy is created tied to the lifetime of the tensor. Uses the default allocator.

Parameters:

env - The current OrtEnvironment.

data - The tensor data.

shape - The shape of tensor.

Returns:

An OnnxTensor of the required shape.

Throws:

OrtException - Thrown if there is an onnx error or if the data and shape don't match.