Package ai.onnxruntime

Class OrtSession

java.lang.Object

ai.onnxruntime.OrtSession

All Implemented Interfaces:

AutoCloseable

public class OrtSession
extends Object
implements AutoCloseable

Wraps an ONNX model and allows inference calls.

Allows the inspection of the model's input and output nodes. Produced by an OrtEnvironment.

Most instance methods throw IllegalStateException if the session is closed and the methods are called.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	OrtSession.Result	An AutoCloseable wrapper around a Map containing OnnxValues.
static class	OrtSession.RunOptions	Used to control logging and termination of a call to run(java.util.Map<java.lang.String, ? extends ai.onnxruntime.OnnxTensorLike>).
static class	OrtSession.SessionOptions	Represents the options used to construct this session.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the session, releasing it's resources.
String	endProfiling()	Ends the profiling session and returns the output of the profiler.
Map<String,NodeInfo>	getInputInfo()	Returns the info objects for the inputs, including their names and types.
Set<String>	getInputNames()	Returns the input names.
OnnxModelMetadata	getMetadata()	Gets the metadata for the currently loaded model.
long	getNumInputs()	Returns the number of inputs this model expects.
long	getNumOutputs()	Returns the number of outputs this model expects.
Map<String,NodeInfo>	getOutputInfo()	Returns the info objects for the outputs, including their names and types.
Set<String>	getOutputNames()	Returns the output names.
long	getProfilingStartTimeInNs()	Returns the timestamp that profiling started in nanoseconds.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs)	Scores an input feed dict, returning the map of all inferred outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, OrtSession.RunOptions runOptions)	Scores an input feed dict, returning the map of all inferred outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, Map<String,? extends OnnxValue> pinnedOutputs)	Scores an input feed dict, returning the map of pinned outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, Set<String> requestedOutputs)	Scores an input feed dict, returning the map of requested inferred outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, Set<String> requestedOutputs, OrtSession.RunOptions runOptions)	Scores an input feed dict, returning the map of requested inferred outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, Set<String> requestedOutputs, Map<String,? extends OnnxValue> pinnedOutputs)	Scores an input feed dict, returning the map of requested and pinned outputs.
OrtSession.Result	run(Map<String,? extends OnnxTensorLike> inputs, Set<String> requestedOutputs, Map<String,? extends OnnxValue> pinnedOutputs, OrtSession.RunOptions runOptions)	Scores an input feed dict, returning the map of requested and pinned outputs.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Details

getNumInputs

public long getNumInputs()

Returns the number of inputs this model expects.

Returns:

The number of inputs.

getNumOutputs

public long getNumOutputs()

Returns the number of outputs this model expects.

Returns:

The number of outputs.

getInputNames

public Set<String> getInputNames()

Returns the input names. The underlying collection is sorted based on the input id number.

Returns:

The input names.

getOutputNames

public Set<String> getOutputNames()

Returns the output names. The underlying collection is sorted based on the output id number.

Returns:

The output names.

getInputInfo

public Map<String,NodeInfo> getInputInfo()
                                  throws OrtException

Returns the info objects for the inputs, including their names and types. The underlying collection is sorted based on the input id number.

Returns:

The input information.

Throws:

OrtException - If there was an error in native code.

getOutputInfo

public Map<String,NodeInfo> getOutputInfo()
                                   throws OrtException

Returns the info objects for the outputs, including their names and types. The underlying collection is sorted based on the output id number.

Returns:

The output information.

Throws:

OrtException - If there was an error in native code.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs)
                      throws OrtException

Scores an input feed dict, returning the map of all inferred outputs.

The outputs are sorted based on their id number.

Parameters:

inputs - The inputs to score.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input names are invalid, or if there are zero or too many inputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 OrtSession.RunOptions runOptions)
                      throws OrtException

Scores an input feed dict, returning the map of all inferred outputs.

The outputs are sorted based on their id number.

Parameters:

inputs - The inputs to score.

runOptions - The RunOptions to control this run.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input names are invalid, or if there are zero or too many inputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 Set<String> requestedOutputs)
                      throws OrtException

Scores an input feed dict, returning the map of requested inferred outputs.

The outputs are sorted based on the supplied set traversal order.

Parameters:

inputs - The inputs to score.

requestedOutputs - The requested outputs.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 Set<String> requestedOutputs,
 OrtSession.RunOptions runOptions)
                      throws OrtException

Scores an input feed dict, returning the map of requested inferred outputs.

The outputs are sorted based on the supplied set traversal order.

Parameters:

inputs - The inputs to score.

requestedOutputs - The requested outputs.

runOptions - The RunOptions to control this run.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 Map<String,? extends OnnxValue> pinnedOutputs)
                      throws OrtException

Scores an input feed dict, returning the map of pinned outputs.

The outputs are sorted based on the supplied map traversal order.

Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

Parameters:

inputs - The inputs to score.

pinnedOutputs - The requested outputs which the user has allocated.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 Set<String> requestedOutputs,
 Map<String,? extends OnnxValue> pinnedOutputs)
                      throws OrtException

Scores an input feed dict, returning the map of requested and pinned outputs.

The outputs are sorted based on the supplied set traversal order with pinned outputs first, then requested outputs. An IllegalArgumentException is thrown if the same output name appears in both the requested outputs and the pinned outputs.

Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

Parameters:

inputs - The inputs to score.

requestedOutputs - The requested outputs which ORT will allocate.

pinnedOutputs - The requested outputs which the user has allocated.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.

run

public OrtSession.Result run(Map<String,? extends OnnxTensorLike> inputs,
 Set<String> requestedOutputs,
 Map<String,? extends OnnxValue> pinnedOutputs,
 OrtSession.RunOptions runOptions)
                      throws OrtException

Scores an input feed dict, returning the map of requested and pinned outputs.

The outputs are sorted based on the supplied set traversal order with pinned outputs first, then requested outputs. An IllegalArgumentException is thrown if the same output name appears in both the requested outputs and the pinned outputs.

Note: pinned outputs are not owned by the OrtSession.Result object, and are not closed when the result object is closed.

Parameters:

inputs - The inputs to score.

requestedOutputs - The requested outputs which ORT will allocate.

pinnedOutputs - The requested outputs which the user has allocated.

runOptions - The RunOptions to control this run.

Returns:

The inferred outputs.

Throws:

OrtException - If there was an error in native code, the input or output names are invalid, or if there are zero or too many inputs or outputs.

getMetadata

public OnnxModelMetadata getMetadata()
                              throws OrtException

Gets the metadata for the currently loaded model.

Returns:

The metadata.

Throws:

OrtException - If the native call failed.

getProfilingStartTimeInNs

public long getProfilingStartTimeInNs()
                               throws OrtException

Returns the timestamp that profiling started in nanoseconds.

Returns:

the profiling start time in ns.

Throws:

OrtException - If the native call failed.

endProfiling

public String endProfiling()
                    throws OrtException

Ends the profiling session and returns the output of the profiler.

Profiling should be enabled in the OrtSession.SessionOptions used to construct this Session.

Returns:

The profiling output.

Throws:

OrtException - If the native call failed.

toString

public String toString()

Overrides:

toString in class Object

close

public void close()
           throws OrtException

Closes the session, releasing it's resources.

Specified by:

close in interface AutoCloseable

Throws:

OrtException - If it failed to close.