A B C D E F G H I J L M N O P R S T U V W

A

acceptInboundMessage(Object) - Method in class ai.djl.serving.http.ConfigurableHttpRequestHandler

acceptInboundMessage(Object) - Method in class ai.djl.serving.http.InferenceRequestHandler

acceptInboundMessage(Object) - Method in class ai.djl.serving.http.ManagementRequestHandler

acceptInboundMessage(Object) - Method in interface ai.djl.serving.plugins.RequestHandler

Returns true if this handler can handle the incoming HTTP request.

addFormData(InterfaceHttpData, Input) - Static method in class ai.djl.serving.util.NettyUtils

Parses form data and added to the Input object.

addJob(Job) - Method in class ai.djl.serving.wlm.ModelManager

Adds an inference job to the job queue.

addModel(String, String) - Method in class ai.djl.serving.http.ListModelsResponse

Adds the model tp the list.

addWorker(int, long, boolean, int) - Method in class ai.djl.serving.http.DescribeModelResponse

Adds worker to the worker list.

ai.djl.serving - package ai.djl.serving

Contains a model server implementation.

ai.djl.serving.http - package ai.djl.serving.http

Contains classes that handles REST API calls.

ai.djl.serving.plugins - package ai.djl.serving.plugins

Contains plugin manager implementation of the model server.

ai.djl.serving.util - package ai.djl.serving.util

Contains utilities used by model server.

ai.djl.serving.wlm - package ai.djl.serving.wlm

Contains classes that manage model lifecycle.

Arguments - Class in ai.djl.serving

A class represents parsed command line arguments.

Arguments(CommandLine) - Constructor for class ai.djl.serving.Arguments

Constructs a new Arguments instance.

B

BadRequestException - Exception in ai.djl.serving.http

Thrown when a bad HTTP request is received.

BadRequestException(String) - Constructor for exception ai.djl.serving.http.BadRequestException

Constructs an BadRequestException with the specified detail message.

BadRequestException(String, Throwable) - Constructor for exception ai.djl.serving.http.BadRequestException

Constructs an BadRequestException with the specified detail message and a root cause.

C

channelRead0(ChannelHandlerContext, FullHttpRequest) - Method in class ai.djl.serving.http.HttpRequestHandler

clean() - Method in class ai.djl.serving.util.Connector

Cleans up the left over resources.

close() - Method in class ai.djl.serving.wlm.ModelInfo

closeOnFlush(Channel) - Static method in class ai.djl.serving.util.NettyUtils

Closes the specified channel after all queued write requests are flushed.

ConfigManager - Class in ai.djl.serving.util

A class that hold configuration information.

ConfigurableHttpRequestHandler - Class in ai.djl.serving.http

HttpRequestHandler that tries to process a http-request using the configured RequestHandlers.

ConfigurableHttpRequestHandler(FolderScanPluginManager) - Constructor for class ai.djl.serving.http.ConfigurableHttpRequestHandler

constructing a ConfigurableHttpRequestHandler.

configureModelBatch(int) - Method in class ai.djl.serving.wlm.ModelInfo

Sets a new batchSize and returns a new configured ModelInfo object.

configurePool(int, int) - Method in class ai.djl.serving.wlm.ModelInfo

Sets new configuration for the workerPool backing this model and returns a new configured ModelInfo object.

Connector - Class in ai.djl.serving.util

A class represents model server's socket listener.

Connector.ConnectorType - Enum in ai.djl.serving.util

An enum represents type of connector.

convertPrivateKey(byte[]) - Static method in class ai.djl.serving.util.OpenSslKey

Convert OpenSSL private key to PKCS8 private key.

D

describeModel(String) - Method in class ai.djl.serving.wlm.ModelManager

Returns a list of worker information for specified model.

DescribeModelResponse - Class in ai.djl.serving.http

A class that holds information about model status.

DescribeModelResponse() - Constructor for class ai.djl.serving.http.DescribeModelResponse

Constructs a DescribeModelResponse instance.

DescribeModelResponse.Worker - Class in ai.djl.serving.http

A class that holds workers information.

dumpConfigurations() - Method in class ai.djl.serving.util.ConfigManager

Prints out this configuration.

E

equals(Object) - Method in class ai.djl.serving.util.Connector

ErrorResponse - Class in ai.djl.serving.http

A class that holds error message.

ErrorResponse(int, String, String) - Constructor for class ai.djl.serving.http.ErrorResponse

Constructs a ErrorResponse instance with code, type and message.

exceptionCaught(ChannelHandlerContext, Throwable) - Method in class ai.djl.serving.http.HttpRequestHandler

F

findImplementations(Class<T>) - Method in class ai.djl.serving.plugins.FolderScanPluginManager

returns a set of plug-in components implementing the specific service interface.

findImplementations(Class<T>) - Method in interface ai.djl.serving.plugins.PluginManager

Returns a set of plug-in components implementing the specific service interface.

FolderScanPluginManager - Class in ai.djl.serving.plugins

The PluginManager is responsible to load and manage plugins from the file system.

FolderScanPluginManager(ConfigManager) - Constructor for class ai.djl.serving.plugins.FolderScanPluginManager

Constructs a PluginManager instance.

G

generate() - Method in class ai.djl.serving.wlm.WorkerIdGenerator

generate a new worker id.

getBatchSize() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the batch size.

getBatchSize() - Method in class ai.djl.serving.util.ConfigManager

Returns the default batchSize for workers.

getBatchSize() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the configured batch size.

getBytes(ByteBuf) - Static method in class ai.djl.serving.util.NettyUtils

Returns the bytes for the specified ByteBuf.

getChildGroup() - Method in class ai.djl.serving.util.ServerGroups

Returns the client event loop group.

getClientChannel() - Method in class ai.djl.serving.util.Connector

Returns the client channel class.

getCode() - Method in class ai.djl.serving.http.ErrorResponse

Returns the error code.

getConfigFile() - Method in class ai.djl.serving.Arguments

Returns the configuration file path.

getConnector(Connector.ConnectorType) - Method in class ai.djl.serving.util.ConfigManager

Returns the models server socket connector.

getCorsAllowedHeaders() - Method in class ai.djl.serving.util.ConfigManager

Returns the CORS allowed headers setting.

getCorsAllowedMethods() - Method in class ai.djl.serving.util.ConfigManager

Returns the CORS allowed method setting.

getCorsAllowedOrigin() - Method in class ai.djl.serving.util.ConfigManager

Returns the CORS allowed origin setting.

getDefaultWorkers() - Method in class ai.djl.serving.util.ConfigManager

Returns the default number of workers for a new registered model.

getId() - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Returns the worker's ID.

getInput() - Method in class ai.djl.serving.wlm.Job

Returns the input data.

getInstance() - Static method in class ai.djl.serving.util.ConfigManager

Returns the singleton ConfigManager instance.

getInstance() - Static method in class ai.djl.serving.wlm.ModelManager

Returns the singleton ModelManager instance.

getIntParameter(QueryStringDecoder, String, int) - Static method in class ai.djl.serving.util.NettyUtils

Read the parameter's integer value for the key from the uri.

getIoRatio() - Method in class ai.djl.serving.util.ConfigManager

Returns the native IO ratio.

getJobQueueSize() - Method in class ai.djl.serving.util.ConfigManager

Returns the default job queue size.

getLoadModels() - Method in class ai.djl.serving.util.ConfigManager

Returns the model urls that to be loaded at startup.

getMaxBatchDelay() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the maximum delay in milliseconds to aggregate a batch.

getMaxBatchDelay() - Method in class ai.djl.serving.util.ConfigManager

Returns the default maxBatchDelay for the working queue.

getMaxBatchDelay() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the maximum delay in milliseconds to aggregate a batch.

getMaxIdleTime() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the maximum idle time for worker threads.

getMaxIdleTime() - Method in class ai.djl.serving.util.ConfigManager

Returns the default max idle time for workers.

getMaxIdleTime() - Method in class ai.djl.serving.wlm.ModelInfo

returns the configured maxIdleTime of workers.

getMaxRequestSize() - Method in class ai.djl.serving.util.ConfigManager

Returns the maximum allowed request size in bytes.

getMaxWorkers() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the desired maximum number of workers.

getMaxWorkers() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the configured maximum number of workers.

getMessage() - Method in class ai.djl.serving.http.ErrorResponse

Returns the error message.

getMinWorkers() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the desired minimum number of workers.

getMinWorkers() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the configured minimum number of workers.

getModel() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the loaded ZooModel.

getModelDir() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the model cache directory.

getModelName() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the model name.

getModelName() - Method in class ai.djl.serving.http.ListModelsResponse.ModelItem

Returns the model name.

getModelName() - Method in class ai.djl.serving.wlm.Job

Returns the model name that associated with this job.

getModelName() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the model name.

getModels() - Method in class ai.djl.serving.Arguments

Returns the model urls that specified in command line.

getModels() - Method in class ai.djl.serving.http.ListModelsResponse

Returns a list of models.

getModels() - Method in class ai.djl.serving.wlm.ModelManager

Returns the registry of all models.

getModelServerHome() - Static method in class ai.djl.serving.util.ConfigManager

Returns the model server home directory.

getModelStore() - Method in class ai.djl.serving.Arguments

Returns the model store location.

getModelStore() - Method in class ai.djl.serving.util.ConfigManager

Returns the model store location.

getModelUrl() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the model URL.

getModelUrl() - Method in class ai.djl.serving.http.ListModelsResponse.ModelItem

Returns the model url.

getModelUrl() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the model url.

getModelUrlPattern() - Method in class ai.djl.serving.util.ConfigManager

Returns the allowed model url pattern regex.

getNettyClientThreads() - Method in class ai.djl.serving.util.ConfigManager

Returns the configured netty client threads.

getNettyThreads() - Method in class ai.djl.serving.util.ConfigManager

Returns the configured netty threads.

getNextPageToken() - Method in class ai.djl.serving.http.ListModelsResponse

Returns the next page token.

getNumberOfGpu() - Method in class ai.djl.serving.util.ConfigManager

Returns the number of GPUs to be used.

getOptions() - Static method in class ai.djl.serving.Arguments

Builds the command line options.

getParameter(QueryStringDecoder, String, String) - Static method in class ai.djl.serving.util.NettyUtils

Reads the parameter's value for the key from the uri.

getPluginFolder() - Method in class ai.djl.serving.util.ConfigManager

return the folder where the model search for plugins.

getProperty(String, String) - Method in class ai.djl.serving.util.ConfigManager

Returns the value with the specified key in this configuration.

getQueueSize() - Method in class ai.djl.serving.wlm.ModelInfo

Returns the configured size of the workers queue.

getRequestId() - Method in class ai.djl.serving.http.Session

Returns the request ID.

getRequestId(Channel) - Static method in class ai.djl.serving.util.NettyUtils

Returns the request ID for the specified channel.

getRequestId() - Method in class ai.djl.serving.wlm.Job

Returns the request id.

getServerChannel() - Method in class ai.djl.serving.util.Connector

Returns the server channel class.

getServerGroup() - Method in class ai.djl.serving.util.ServerGroups

Returns the server event loop group.

getSocketAddress() - Method in class ai.djl.serving.util.Connector

Returns the TCP socket listening address.

getSocketPath() - Method in class ai.djl.serving.util.Connector

Returns the unix domain socket path.

getSocketType() - Method in class ai.djl.serving.util.Connector

Returns the socket type.

getSslContext() - Method in class ai.djl.serving.util.ConfigManager

Returns a SSLContext instance.

getStartTime() - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Returns the worker's start time.

getStartupModels() - Method in class ai.djl.serving.wlm.ModelManager

Returns a set of models that was loaded at startup.

getStatus() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns the model's status.

getStatus() - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Returns the worker's status.

getStatus() - Method in class ai.djl.serving.http.StatusResponse

Returns the status.

getType() - Method in class ai.djl.serving.http.ErrorResponse

Returns the error type.

getType() - Method in class ai.djl.serving.util.Connector

Returns the type of the connector.

getWorkers() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns all workers information of the model.

GpuAssignmentStrategy - Interface in ai.djl.serving.wlm

Strategies to find next available GpuID.

H

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in class ai.djl.serving.http.ConfigurableHttpRequestHandler

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in class ai.djl.serving.http.HttpRequestHandler

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in class ai.djl.serving.http.InferenceRequestHandler

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in class ai.djl.serving.http.InvalidRequestHandler

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in class ai.djl.serving.http.ManagementRequestHandler

handleRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder, String[]) - Method in interface ai.djl.serving.plugins.RequestHandler

The main method which handles request.

hashCode() - Method in class ai.djl.serving.util.Connector

hasHelp() - Method in class ai.djl.serving.Arguments

Returns if the command line has help option.

HttpRequestHandler - Class in ai.djl.serving.http

A class handling inbound HTTP requests.

HttpRequestHandler() - Constructor for class ai.djl.serving.http.HttpRequestHandler

I

InferenceRequestHandler - Class in ai.djl.serving.http

A class handling inbound HTTP requests for the management API.

InferenceRequestHandler() - Constructor for class ai.djl.serving.http.InferenceRequestHandler

default constructor.

inferModelNameFromUrl(String) - Static method in class ai.djl.serving.wlm.ModelInfo

Infer model name form model URL in case model name is not provided.

init(Arguments) - Static method in class ai.djl.serving.util.ConfigManager

Initialize the global ConfigManager instance.

init(ConfigManager) - Static method in class ai.djl.serving.wlm.ModelManager

Initialized the global ModelManager instance.

initChannel(Channel) - Method in class ai.djl.serving.ServerInitializer

InternalServerException - Exception in ai.djl.serving.http

Thrown when an internal server failure occurs.

InternalServerException(String) - Constructor for exception ai.djl.serving.http.InternalServerException

Constructs an InternalServerException with the specified detail message.

InternalServerException(String, Throwable) - Constructor for exception ai.djl.serving.http.InternalServerException

Constructs an BadRequestException with the specified detail message and cause.

InvalidRequestHandler - Class in ai.djl.serving.http

A class handling unhandled inbound HTTP requests.

InvalidRequestHandler() - Constructor for class ai.djl.serving.http.InvalidRequestHandler

isDebug() - Method in class ai.djl.serving.util.ConfigManager

Returns if debug is enabled.

isFinished() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.

isFinished() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator

Checks if this BatchAggregator and the thread can be shutdown or if this aggregator waits for more data.

isGpu() - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Return if the worker using GPU.

isLoadedAtStartup() - Method in class ai.djl.serving.http.DescribeModelResponse

Returns if the models was loaded at startup.

isRunning() - Method in class ai.djl.serving.ModelServer

Return if the server is running.

isSsl() - Method in class ai.djl.serving.util.Connector

Return if the connector requires SSL.

isUds() - Method in class ai.djl.serving.util.Connector

Returns if the connector is using unix domain socket.

J

Job - Class in ai.djl.serving.wlm

A class represents an inference job.

Job(ChannelHandlerContext, String, Input) - Constructor for class ai.djl.serving.wlm.Job

Constructs an new Job instance.

L

ListModelsResponse - Class in ai.djl.serving.http

A class that holds information about the current registered models.

ListModelsResponse() - Constructor for class ai.djl.serving.http.ListModelsResponse

Constructs a new ListModelsResponse instance.

ListModelsResponse.ModelItem - Class in ai.djl.serving.http

A class that holds model name and url.

loadPlugins() - Method in class ai.djl.serving.plugins.FolderScanPluginManager

Loads all plugins from the plugin folder and register them.

M

main(String[]) - Static method in class ai.djl.serving.ModelServer

The entry point for the model server.

ManagementRequestHandler - Class in ai.djl.serving.http

A class handling inbound HTTP requests to the management API.

ManagementRequestHandler() - Constructor for class ai.djl.serving.http.ManagementRequestHandler

MethodNotAllowedException - Exception in ai.djl.serving.http

Thrown when a HTTP request which method is not allowed.

MethodNotAllowedException() - Constructor for exception ai.djl.serving.http.MethodNotAllowedException

Constructs an MethodNotAllowedException with null as its error detail message.

ModelInfo - Class in ai.djl.serving.wlm

A class represent a loaded model and it's metadata.

ModelInfo(String, String, ZooModel<Input, Output>, int, int, int, int) - Constructor for class ai.djl.serving.wlm.ModelInfo

Constructs a new ModelInfo instance.

ModelItem() - Constructor for class ai.djl.serving.http.ListModelsResponse.ModelItem

Constructs a new ModelItem instance.

ModelItem(String, String) - Constructor for class ai.djl.serving.http.ListModelsResponse.ModelItem

Constructs a new ModelItem instance with model name and url.

ModelManager - Class in ai.djl.serving.wlm

A class that in charge of managing models.

ModelServer - Class in ai.djl.serving

The main entry point for model server.

ModelServer(ConfigManager) - Constructor for class ai.djl.serving.ModelServer

Creates a new ModelServer instance.

N

NettyUtils - Class in ai.djl.serving.util

A utility class that handling Netty request and response.

newEventLoopGroup(int) - Static method in class ai.djl.serving.util.Connector

Creates a new netty EventLoopGroup.

nextGpuId() - Method in interface ai.djl.serving.wlm.GpuAssignmentStrategy

Returns next gpuId.

nextGpuId() - Method in class ai.djl.serving.wlm.RoundRobinGpuAssignmentStrategy

Returns next gpuId.

O

OpenSslKey - Class in ai.djl.serving.util

A utility class converting OpenSSL private key to PKCS8 private key.

P

parse(String, Connector.ConnectorType) - Static method in class ai.djl.serving.util.Connector

Create a Connector instance based on binding string.

parseRequest(ChannelHandlerContext, FullHttpRequest, QueryStringDecoder) - Method in class ai.djl.serving.http.RequestParser

parsing a request.

PermanentBatchAggregator - Class in ai.djl.serving.wlm

a batch aggregator that never terminates by itself.

PermanentBatchAggregator(ModelInfo, LinkedBlockingDeque<Job>) - Constructor for class ai.djl.serving.wlm.PermanentBatchAggregator

Constructs a PermanentBatchAggregator instance.

PluginManager - Interface in ai.djl.serving.plugins

The Plugin Manager is responsible to load and manage plugins from the filesystem.

pollBatch() - Method in class ai.djl.serving.wlm.PermanentBatchAggregator

Fills in the list with a batch of jobs.

pollBatch() - Method in class ai.djl.serving.wlm.TemporaryBatchAggregator

Fills in the list with a batch of jobs.

R

registerChannel(Channel) - Method in class ai.djl.serving.util.ServerGroups

Registers a channel to be tracked.

registerModel(String, String, int, int, int) - Method in class ai.djl.serving.wlm.ModelManager

Registers and loads a model.

RequestHandler<T> - Interface in ai.djl.serving.plugins

Interface to be implemented by HtttpRequestHandler.

RequestParser - Class in ai.djl.serving.http

a parser for inbound request.

RequestParser() - Constructor for class ai.djl.serving.http.RequestParser

requestReceived(Channel, HttpRequest) - Static method in class ai.djl.serving.util.NettyUtils

Updates session when a HTTP request is received.

reset() - Method in class ai.djl.serving.util.ServerGroups

Resets the ServerGroups.

ResourceNotFoundException - Exception in ai.djl.serving.http

Thrown when a HTTP request what requested resource is not found.

ResourceNotFoundException() - Constructor for exception ai.djl.serving.http.ResourceNotFoundException

Constructs an ResourceNotFoundException with null as its error detail message.

ResourceNotFoundException(Throwable) - Constructor for exception ai.djl.serving.http.ResourceNotFoundException

Constructs an ResourceNotFoundException with a root cause.

RoundRobinGpuAssignmentStrategy - Class in ai.djl.serving.wlm

Assign next gpu using round robin to get the next gpuID.

RoundRobinGpuAssignmentStrategy(ConfigManager) - Constructor for class ai.djl.serving.wlm.RoundRobinGpuAssignmentStrategy

Construct a round robin gpu assignment strategy.

S

ScaleCapacityExceededException - Exception in ai.djl.serving.wlm

Is thrown when capacity of workers is reached during autoscaling.

ScaleCapacityExceededException() - Constructor for exception ai.djl.serving.wlm.ScaleCapacityExceededException

No arguments.

ScaleCapacityExceededException(String) - Constructor for exception ai.djl.serving.wlm.ScaleCapacityExceededException

construct using a message.

ScaleCapacityExceededException(Throwable) - Constructor for exception ai.djl.serving.wlm.ScaleCapacityExceededException

construct using a cause.

ScaleCapacityExceededException(String, Throwable) - Constructor for exception ai.djl.serving.wlm.ScaleCapacityExceededException

construct using a message and a clause.

ScaleCapacityExceededException(String, Throwable, boolean, boolean) - Constructor for exception ai.djl.serving.wlm.ScaleCapacityExceededException

construct using a message cause and flags.

scaleWorkers(int, int) - Method in class ai.djl.serving.wlm.ModelInfo

Sets new workers capcities for this model and returns a new configured ModelInfo object.

sendError(ChannelHandlerContext, Throwable) - Static method in class ai.djl.serving.util.NettyUtils

Sends error to client with exception.

sendError(ChannelHandlerContext, HttpResponseStatus, Throwable) - Static method in class ai.djl.serving.util.NettyUtils

Sends error to client with HTTP status and exception.

sendError(HttpResponseStatus, String) - Method in class ai.djl.serving.wlm.Job

Sends error to the client.

sendHttpResponse(ChannelHandlerContext, FullHttpResponse, boolean) - Static method in class ai.djl.serving.util.NettyUtils

Send HTTP response to client.

sendJsonResponse(ChannelHandlerContext, Object) - Static method in class ai.djl.serving.util.NettyUtils

Sends the json object to client.

sendJsonResponse(ChannelHandlerContext, Object, HttpResponseStatus) - Static method in class ai.djl.serving.util.NettyUtils

Sends the json string to client with specified status.

sendJsonResponse(ChannelHandlerContext, String) - Static method in class ai.djl.serving.util.NettyUtils

Sends the json string to client.

sendJsonResponse(ChannelHandlerContext, String, HttpResponseStatus) - Static method in class ai.djl.serving.util.NettyUtils

Sends the json object to client with specified status.

sendOutput(Output) - Method in class ai.djl.serving.wlm.Job

Sends the response back to the client.

ServerGroups - Class in ai.djl.serving.util

A class that holds all opened ChannelGroup and EventLoopGroup.

ServerGroups(ConfigManager) - Constructor for class ai.djl.serving.util.ServerGroups

Constructs a new ServerGroups instance.

ServerInitializer - Class in ai.djl.serving

A special ChannelInboundHandler which offers an easy way to initialize a Channel once it was registered to its EventLoop.

ServerInitializer(SslContext, Connector.ConnectorType, FolderScanPluginManager) - Constructor for class ai.djl.serving.ServerInitializer

Creates a new HttpRequestHandler instance.

ServiceUnavailableException - Exception in ai.djl.serving.http

Thrown when the server is unable to serve the HTTP request.

ServiceUnavailableException(String) - Constructor for exception ai.djl.serving.http.ServiceUnavailableException

Constructs an ServiceUnavailableException with the specified detail message.

Session - Class in ai.djl.serving.http

A class that holds HTTP session information.

Session(String, HttpRequest) - Constructor for class ai.djl.serving.http.Session

Constructs a new Session instance.

setBatchSize(int) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the batch size.

setCode(int) - Method in class ai.djl.serving.http.Session

Sets the HTTP response code.

setGpu(boolean) - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Sets if the worker using GPU.

setId(int) - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Sets the worker's ID.

setLoadedAtStartup(boolean) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the load at startup status.

setMaxBatchDelay(int) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the maximum delay in milliseconds to aggregate a batch.

setMaxIdleTime(int) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the max idle time for worker threads.

setMaxWorkers(int) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the desired maximum number of workers.

setMinWorkers(int) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the desired minimum number of workers.

setModelName(String) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the model name.

setModelUrl(String) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the model URL.

setNextPageToken(String) - Method in class ai.djl.serving.http.ListModelsResponse

Sets the next page token.

setScheduled() - Method in class ai.djl.serving.wlm.Job

Marks the job has been scheduled.

setStartTime(Date) - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Sets the worker's start time.

setStatus(String) - Method in class ai.djl.serving.http.DescribeModelResponse

Sets the model's status.

setStatus(String) - Method in class ai.djl.serving.http.DescribeModelResponse.Worker

Sets the worker's status.

shutdown(boolean) - Method in class ai.djl.serving.util.ServerGroups

Shutdowns all opened channels and event loops.

start() - Method in class ai.djl.serving.ModelServer

Main Method that prepares the future for the channel and sets up the ServerBootstrap.

startAndWait() - Method in class ai.djl.serving.ModelServer

Starts the model server and block until server stops.

StatusResponse - Class in ai.djl.serving.http

A class that holds model server status.

StatusResponse() - Constructor for class ai.djl.serving.http.StatusResponse

Constructs a new StatusResponse instance.

StatusResponse(String) - Constructor for class ai.djl.serving.http.StatusResponse

Constructs a new StatusResponse instance with status line.

stop() - Method in class ai.djl.serving.ModelServer

Stops the model server.

T

TemporaryBatchAggregator - Class in ai.djl.serving.wlm

a batch aggregator that terminates after a maximum idle time.

TemporaryBatchAggregator(ModelInfo, LinkedBlockingDeque<Job>) - Constructor for class ai.djl.serving.wlm.TemporaryBatchAggregator

a batch aggregator that terminates after a maximum idle time.

toString() - Method in class ai.djl.serving.http.Session

toString() - Method in class ai.djl.serving.util.Connector

triggerModelUpdated(ModelInfo) - Method in class ai.djl.serving.wlm.ModelManager

trigger that a ModelInfo has been updated.

U

unregisterModel(String) - Method in class ai.djl.serving.wlm.ModelManager

Unregisters a model by its name.

useNativeIo() - Method in class ai.djl.serving.util.ConfigManager

Returns if use netty native IO.

V

valueOf(String) - Static method in enum ai.djl.serving.util.Connector.ConnectorType

Returns the enum constant of this type with the specified name.

valueOf(String) - Static method in enum ai.djl.serving.wlm.WorkerState

Returns the enum constant of this type with the specified name.

values() - Static method in enum ai.djl.serving.util.Connector.ConnectorType

Returns an array containing the constants of this enum type, in the order they are declared.

values() - Static method in enum ai.djl.serving.wlm.WorkerState

Returns an array containing the constants of this enum type, in the order they are declared.

W

Worker() - Constructor for class ai.djl.serving.http.DescribeModelResponse.Worker

WorkerIdGenerator - Class in ai.djl.serving.wlm

class to generate an unique worker id.

WorkerIdGenerator() - Constructor for class ai.djl.serving.wlm.WorkerIdGenerator

WorkerState - Enum in ai.djl.serving.wlm

An enum represents state of a worker.

workerStatus() - Method in class ai.djl.serving.wlm.ModelManager

Sends model server health status to client.

A B C D E F G H I J L M N O P R S T U V W