public final class ModelManager
extends java.lang.Object
| Modifier and Type | Method and Description |
|---|---|
boolean |
addJob(Job job)
Adds an inference job to the job queue.
|
DescribeModelResponse |
describeModel(java.lang.String modelName)
Returns a list of worker information for specified model.
|
static ModelManager |
getInstance()
Returns the singleton
ModelManager instance. |
java.util.Map<java.lang.String,ModelInfo> |
getModels()
Returns the registry of all models.
|
java.util.Set<java.lang.String> |
getStartupModels()
Returns a set of models that was loaded at startup.
|
static void |
init(ConfigManager configManager)
Initialized the global
ModelManager instance. |
java.util.concurrent.CompletableFuture<ModelInfo> |
registerModel(java.lang.String modelName,
java.lang.String modelUrl,
int batchSize,
int maxBatchDelay,
int maxIdleTime)
Registers and loads a model.
|
void |
triggerModelUpdated(ModelInfo modelInfo)
trigger that a ModelInfo has been updated.
|
boolean |
unregisterModel(java.lang.String modelName)
Unregisters a model by its name.
|
java.util.concurrent.CompletableFuture<java.lang.String> |
workerStatus()
Sends model server health status to client.
|
public static void init(ConfigManager configManager)
ModelManager instance.configManager - the configurationpublic static ModelManager getInstance()
ModelManager instance.ModelManager instancepublic java.util.concurrent.CompletableFuture<ModelInfo> registerModel(java.lang.String modelName, java.lang.String modelUrl, int batchSize, int maxBatchDelay, int maxIdleTime)
modelName - the name of the model for HTTP endpointmodelUrl - the model urlbatchSize - the batch sizemaxBatchDelay - the maximum delay for batchingmaxIdleTime - the maximum idle time of the worker threads before scaling down.CompletableFuture instancepublic boolean unregisterModel(java.lang.String modelName)
modelName - the model name to be unregisteredtrue if unregister successpublic void triggerModelUpdated(ModelInfo modelInfo)
modelInfo - the model that has been updatedpublic java.util.Map<java.lang.String,ModelInfo> getModels()
public java.util.Set<java.lang.String> getStartupModels()
public boolean addJob(Job job) throws ai.djl.repository.zoo.ModelNotFoundException
job - an inference job to be executedtrue if submit successai.djl.repository.zoo.ModelNotFoundException - if the model is not registeredpublic DescribeModelResponse describeModel(java.lang.String modelName) throws ai.djl.repository.zoo.ModelNotFoundException
modelName - the model to be queriedai.djl.repository.zoo.ModelNotFoundException - if specified model not foundpublic java.util.concurrent.CompletableFuture<java.lang.String> workerStatus()