Package ai.onnxruntime

Class OrtEnvironment.ThreadingOptions

java.lang.Object

ai.onnxruntime.OrtEnvironment.ThreadingOptions

All Implemented Interfaces:

AutoCloseable

Enclosing class:

OrtEnvironment

public static final class OrtEnvironment.ThreadingOptions
extends Object
implements AutoCloseable

Controls the global thread pools in the environment. Only used if the session is constructed using an options with OrtSession.SessionOptions.disablePerSessionThreads() set.

Constructor Summary

Constructors

Constructor	Description
ThreadingOptions()	Create an empty threading options.

Method Summary

Modifier and Type	Method	Description
void	close()	Closes the threading options.
void	setGlobalDenormalAsZero()	When this is set it causes intra-op and inter-op thread pools to flush denormal values to zero.
void	setGlobalInterOpNumThreads(int numThreads)	Sets the number of threads available for inter-op parallelism (i.e.
void	setGlobalIntraOpNumThreads(int numThreads)	Sets the number of threads available for intra-op parallelism (i.e.
void	setGlobalSpinControl(boolean allowSpinning)	Allows spinning of thread pools when their queues are empty.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

ThreadingOptions

public ThreadingOptions()

Create an empty threading options.

Method Details

close

public void close()

Closes the threading options.

Specified by:

close in interface AutoCloseable

setGlobalInterOpNumThreads

public void setGlobalInterOpNumThreads(int numThreads)
                                throws OrtException

Sets the number of threads available for inter-op parallelism (i.e. running multiple ops in parallel).

Setting it to 0 will allow ORT to choose the number of threads, setting it to 1 will cause the main thread to be used (i.e., no thread pools will be used).

Parameters:

numThreads - The number of threads.

Throws:

OrtException - If there was an error in native code.

setGlobalIntraOpNumThreads

public void setGlobalIntraOpNumThreads(int numThreads)
                                throws OrtException

Sets the number of threads available for intra-op parallelism (i.e. within a single op).

Setting it to 0 will allow ORT to choose the number of threads, setting it to 1 will cause the main thread to be used (i.e., no thread pools will be used).

Parameters:

numThreads - The number of threads.

Throws:

OrtException - If there was an error in native code.

setGlobalSpinControl

public void setGlobalSpinControl(boolean allowSpinning)
                          throws OrtException

Allows spinning of thread pools when their queues are empty. This call sets the value for both inter-op and intra-op thread pools.

If the CPU usage is very high then do not enable this.

Parameters:

allowSpinning - If true allow the thread pools to spin.

Throws:

OrtException - If there was an error in native code.

setGlobalDenormalAsZero

public void setGlobalDenormalAsZero()
                             throws OrtException

When this is set it causes intra-op and inter-op thread pools to flush denormal values to zero.

Throws:

OrtException - If there was an error in native code.