public static interface MultiModelConfig.Builder extends SdkPojo, CopyableBuilder<MultiModelConfig.Builder,MultiModelConfig>
| Modifier and Type | Method and Description |
|---|---|
MultiModelConfig.Builder |
modelCacheSetting(ModelCacheSetting modelCacheSetting)
Whether to cache models for a multi-model endpoint.
|
MultiModelConfig.Builder |
modelCacheSetting(String modelCacheSetting)
Whether to cache models for a multi-model endpoint.
|
equalsBySdkFields, sdkFieldscopyapplyMutation, buildMultiModelConfig.Builder modelCacheSetting(String modelCacheSetting)
Whether to cache models for a multi-model endpoint. By default, multi-model endpoints cache models so that a
model does not have to be loaded into memory each time it is invoked. Some use cases do not benefit from
model caching. For example, if an endpoint hosts a large number of models that are each invoked infrequently,
the endpoint might perform better if you disable model caching. To disable model caching, set the value of
this parameter to Disabled.
modelCacheSetting - Whether to cache models for a multi-model endpoint. By default, multi-model endpoints cache models so
that a model does not have to be loaded into memory each time it is invoked. Some use cases do not
benefit from model caching. For example, if an endpoint hosts a large number of models that are each
invoked infrequently, the endpoint might perform better if you disable model caching. To disable model
caching, set the value of this parameter to Disabled.ModelCacheSetting,
ModelCacheSettingMultiModelConfig.Builder modelCacheSetting(ModelCacheSetting modelCacheSetting)
Whether to cache models for a multi-model endpoint. By default, multi-model endpoints cache models so that a
model does not have to be loaded into memory each time it is invoked. Some use cases do not benefit from
model caching. For example, if an endpoint hosts a large number of models that are each invoked infrequently,
the endpoint might perform better if you disable model caching. To disable model caching, set the value of
this parameter to Disabled.
modelCacheSetting - Whether to cache models for a multi-model endpoint. By default, multi-model endpoints cache models so
that a model does not have to be loaded into memory each time it is invoked. Some use cases do not
benefit from model caching. For example, if an endpoint hosts a large number of models that are each
invoked infrequently, the endpoint might perform better if you disable model caching. To disable model
caching, set the value of this parameter to Disabled.ModelCacheSetting,
ModelCacheSettingCopyright © 2022. All rights reserved.