Package ai.djl.translate

Interface ServingTranslator

All Superinterfaces:

PostProcessor<Output>, PreProcessor<Input>, StreamingTranslator<Input,Output>, Translator<Input,Output>

public interface ServingTranslator
extends StreamingTranslator<Input,Output>

A Translator that can handle generic Input and Output.

Nested Class Summary

Nested classes/interfaces inherited from interface ai.djl.inference.streaming.StreamingTranslator

StreamingTranslator.StreamOutput<O>, StreamingTranslator.Support

Method Summary

Modifier and Type	Method	Description
default StreamingTranslator.Support	getSupport()	Returns what kind of StreamingTranslator.StreamOutput this StreamingTranslator supports.
default StreamingTranslator.StreamOutput<Output>	processStreamOutput(TranslatorContext ctx, java.util.stream.Stream<NDList> list)	Processes the output NDList to the corresponding output object.
void	setArguments(java.util.Map<java.lang.String,?> arguments)	Sets the configurations for the Translator instance.

Methods inherited from interface ai.djl.translate.PostProcessor

processOutput

Methods inherited from interface ai.djl.translate.PreProcessor

processInput

Methods inherited from interface ai.djl.inference.streaming.StreamingTranslator

supportsAsync, supportsIterative

Methods inherited from interface ai.djl.translate.Translator

getBatchifier, getExpansions, prepare, toBatchTranslator, toBatchTranslator

Method Detail

setArguments

void setArguments(java.util.Map<java.lang.String,?> arguments)

Sets the configurations for the Translator instance.

Parameters:

arguments - the configurations for the Translator instance

getSupport

default StreamingTranslator.Support getSupport()

Returns what kind of StreamingTranslator.StreamOutput this StreamingTranslator supports.

Specified by:

getSupport in interface StreamingTranslator<Input,Output>

Returns:

what kind of StreamingTranslator.StreamOutput this StreamingTranslator supports

processStreamOutput

default StreamingTranslator.StreamOutput<Output> processStreamOutput(TranslatorContext ctx,
                                                                     java.util.stream.Stream<NDList> list)

Processes the output NDList to the corresponding output object.

Specified by:

processStreamOutput in interface StreamingTranslator<Input,Output>

Parameters:

ctx - the toolkit used for post-processing

list - the output NDList after inference, usually immutable in engines like PyTorch. @see Issue 1774

Returns:

the output object of expected type