scaleway_async.inference.v1 package

Submodules

scaleway_async.inference.v1.api module

class scaleway_async.inference.v1.api.InferenceV1API(client: Client, *, bypass_validation: bool = False)

Bases: API

This API allows you to handle your Managed Inference services.

async create_deployment(*, model_id: str, node_type_name: str, endpoints: List[EndpointSpec], region: Optional[str] = None, name: Optional[str] = None, project_id: Optional[str] = None, accept_eula: Optional[bool] = None, tags: Optional[List[str]] = None, min_size: Optional[int] = None, max_size: Optional[int] = None, quantization: Optional[DeploymentQuantization] = None) → Deployment

Create a deployment. Create a new inference deployment related to a specific model. :param model_id: ID of the model to use. :param node_type_name: Name of the node type to use. :param endpoints: List of endpoints to create. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the deployment. :param project_id: ID of the Project to create the deployment in. :param accept_eula: If the model has an EULA, you must accept it before proceeding. The terms of the EULA can be retrieved using the GetModelEula API call. :param tags: List of tags to apply to the deployment. :param min_size: Defines the minimum size of the pool. :param max_size: Defines the maximum size of the pool. :param quantization: Quantization settings to apply to this deployment. :return: Deployment

Usage:

result = await api.create_deployment(
    model_id="example",
    node_type_name="example",
    endpoints=[],
)

async create_endpoint(*, deployment_id: str, endpoint: EndpointSpec, region: Optional[str] = None) → Endpoint

Create an endpoint. Create a new Endpoint related to a specific deployment. :param deployment_id: ID of the deployment to create the endpoint for. :param endpoint: Specification of the endpoint. :param region: Region to target. If none is passed will use default region from the config. :return: Endpoint

Usage:

result = await api.create_endpoint(
    deployment_id="example",
    endpoint=EndpointSpec(),
)

async create_model(*, source: ModelSource, region: Optional[str] = None, name: Optional[str] = None, project_id: Optional[str] = None) → Model

Import a model. Import a new model to your model library. :param source: Where to import the model from. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the model. :param project_id: ID of the Project to import the model in. :return: Model

Usage:

result = await api.create_model(
    source=ModelSource(),
)

async delete_deployment(*, deployment_id: str, region: Optional[str] = None) → Deployment

Delete a deployment. Delete an existing inference deployment. :param deployment_id: ID of the deployment to delete. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.delete_deployment(
    deployment_id="example",
)

async delete_endpoint(*, endpoint_id: str, region: Optional[str] = None) → None

Delete an endpoint. Delete an existing Endpoint. :param endpoint_id: ID of the endpoint to delete. :param region: Region to target. If none is passed will use default region from the config.

Usage:

result = await api.delete_endpoint(
    endpoint_id="example",
)

async delete_model(*, model_id: str, region: Optional[str] = None) → None

Delete a model. Delete an existing model from your model library. :param model_id: ID of the model to delete. :param region: Region to target. If none is passed will use default region from the config.

Usage:

result = await api.delete_model(
    model_id="example",
)

async get_deployment(*, deployment_id: str, region: Optional[str] = None) → Deployment

Get a deployment. Get the deployment for the given ID. :param deployment_id: ID of the deployment to get. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.get_deployment(
    deployment_id="example",
)

async get_deployment_certificate(*, deployment_id: str, region: Optional[str] = None) → ScwFile

Get the CA certificate. Get the CA certificate used for the deployment of private endpoints. The CA certificate will be returned as a PEM file. :param deployment_id: :param region: Region to target. If none is passed will use default region from the config. :return: ScwFile

Usage:

result = await api.get_deployment_certificate(
    deployment_id="example",
)

async get_model(*, model_id: str, region: Optional[str] = None) → Model

Get a model. Get the model for the given ID. :param model_id: ID of the model to get. :param region: Region to target. If none is passed will use default region from the config. :return: Model

Usage:

result = await api.get_model(
    model_id="example",
)

async list_deployments(*, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None, order_by: Optional[ListDeploymentsRequestOrderBy] = None, project_id: Optional[str] = None, organization_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → ListDeploymentsResponse

List inference deployments. List all your inference deployments. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of deployments to return per page. :param order_by: Order in which to return results. :param project_id: Filter by Project ID. :param organization_id: Filter by Organization ID. :param name: Filter by deployment name. :param tags: Filter by tags. :return: ListDeploymentsResponse

Usage:

result = await api.list_deployments()

async list_deployments_all(*, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None, order_by: Optional[ListDeploymentsRequestOrderBy] = None, project_id: Optional[str] = None, organization_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → List[Deployment]

List inference deployments. List all your inference deployments. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of deployments to return per page. :param order_by: Order in which to return results. :param project_id: Filter by Project ID. :param organization_id: Filter by Organization ID. :param name: Filter by deployment name. :param tags: Filter by tags. :return: List[Deployment]

Usage:

result = await api.list_deployments_all()

async list_models(*, region: Optional[str] = None, order_by: Optional[ListModelsRequestOrderBy] = None, page: Optional[int] = None, page_size: Optional[int] = None, project_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → ListModelsResponse

List models. List all available models. :param region: Region to target. If none is passed will use default region from the config. :param order_by: Order in which to return results. :param page: Page number to return. :param page_size: Maximum number of models to return per page. :param project_id: Filter by Project ID. :param name: Filter by model name. :param tags: Filter by tags. :return: ListModelsResponse

Usage:

result = await api.list_models()

async list_models_all(*, region: Optional[str] = None, order_by: Optional[ListModelsRequestOrderBy] = None, page: Optional[int] = None, page_size: Optional[int] = None, project_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → List[Model]

List models. List all available models. :param region: Region to target. If none is passed will use default region from the config. :param order_by: Order in which to return results. :param page: Page number to return. :param page_size: Maximum number of models to return per page. :param project_id: Filter by Project ID. :param name: Filter by model name. :param tags: Filter by tags. :return: List[Model]

Usage:

result = await api.list_models_all()

async list_node_types(*, include_disabled_types: bool, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None) → ListNodeTypesResponse

List available node types. List all available node types. By default, the node types returned in the list are ordered by creation date in ascending order, though this can be modified via the order_by field. :param include_disabled_types: Include disabled node types in the response. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of node types to return per page. :return: ListNodeTypesResponse

Usage:

result = await api.list_node_types(
    include_disabled_types=False,
)

async list_node_types_all(*, include_disabled_types: bool, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None) → List[NodeType]

List available node types. List all available node types. By default, the node types returned in the list are ordered by creation date in ascending order, though this can be modified via the order_by field. :param include_disabled_types: Include disabled node types in the response. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of node types to return per page. :return: List[NodeType]

Usage:

result = await api.list_node_types_all(
    include_disabled_types=False,
)

async update_deployment(*, deployment_id: str, region: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None, min_size: Optional[int] = None, max_size: Optional[int] = None, model_id: Optional[str] = None, quantization: Optional[DeploymentQuantization] = None) → Deployment

Update a deployment. Update an existing inference deployment. :param deployment_id: ID of the deployment to update. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the deployment. :param tags: List of tags to apply to the deployment. :param min_size: Defines the new minimum size of the pool. :param max_size: Defines the new maximum size of the pool. :param model_id: Id of the model to set to the deployment. :param quantization: Quantization to use to the deployment. :return: Deployment

Usage:

result = await api.update_deployment(
    deployment_id="example",
)

async update_endpoint(*, endpoint_id: str, region: Optional[str] = None, disable_auth: Optional[bool] = None) → Endpoint

Update an endpoint. Update an existing Endpoint. :param endpoint_id: ID of the endpoint to update. :param region: Region to target. If none is passed will use default region from the config. :param disable_auth: By default, deployments are protected by IAM authentication. When setting this field to true, the authentication will be disabled. :return: Endpoint

Usage:

result = await api.update_endpoint(
    endpoint_id="example",
)

async wait_for_deployment(*, deployment_id: str, region: Optional[str] = None, options: Optional[WaitForOptions[Deployment, Union[bool, Awaitable[bool]]]] = None) → Deployment

Get a deployment. Get the deployment for the given ID. :param deployment_id: ID of the deployment to get. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.get_deployment(
    deployment_id="example",
)

async wait_for_model(*, model_id: str, region: Optional[str] = None, options: Optional[WaitForOptions[Model, Union[bool, Awaitable[bool]]]] = None) → Model

Get a model. Get the model for the given ID. :param model_id: ID of the model to get. :param region: Region to target. If none is passed will use default region from the config. :return: Model

Usage:

result = await api.get_model(
    model_id="example",
)

scaleway_async.inference.v1.content module

scaleway_async.inference.v1.content.DEPLOYMENT_TRANSIENT_STATUSES: List[DeploymentStatus] = [<DeploymentStatus.CREATING: 'creating'>, <DeploymentStatus.DEPLOYING: 'deploying'>, <DeploymentStatus.DELETING: 'deleting'>]

Lists transient statutes of the enum DeploymentStatus.

scaleway_async.inference.v1.content.MODEL_TRANSIENT_STATUSES: List[ModelStatus] = [<ModelStatus.PREPARING: 'preparing'>, <ModelStatus.DOWNLOADING: 'downloading'>]

Lists transient statutes of the enum ModelStatus.

scaleway_async.inference.v1.marshalling module

scaleway_async.inference.v1.marshalling.marshal_CreateDeploymentRequest(request: CreateDeploymentRequest, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_CreateEndpointRequest(request: CreateEndpointRequest, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_CreateModelRequest(request: CreateModelRequest, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_DeploymentQuantization(request: DeploymentQuantization, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_EndpointPrivateNetworkDetails(request: EndpointPrivateNetworkDetails, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_EndpointPublicNetworkDetails(request: EndpointPublicNetworkDetails, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_EndpointSpec(request: EndpointSpec, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_ModelSource(request: ModelSource, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_UpdateDeploymentRequest(request: UpdateDeploymentRequest, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.marshal_UpdateEndpointRequest(request: UpdateEndpointRequest, defaults: ProfileDefaults) → Dict[str, Any]

scaleway_async.inference.v1.marshalling.unmarshal_Deployment(data: Any) → Deployment

scaleway_async.inference.v1.marshalling.unmarshal_DeploymentQuantization(data: Any) → DeploymentQuantization

scaleway_async.inference.v1.marshalling.unmarshal_Endpoint(data: Any) → Endpoint

scaleway_async.inference.v1.marshalling.unmarshal_EndpointPrivateNetworkDetails(data: Any) → EndpointPrivateNetworkDetails

scaleway_async.inference.v1.marshalling.unmarshal_EndpointPublicNetworkDetails(data: Any) → EndpointPublicNetworkDetails

scaleway_async.inference.v1.marshalling.unmarshal_ListDeploymentsResponse(data: Any) → ListDeploymentsResponse

scaleway_async.inference.v1.marshalling.unmarshal_ListModelsResponse(data: Any) → ListModelsResponse

scaleway_async.inference.v1.marshalling.unmarshal_ListNodeTypesResponse(data: Any) → ListNodeTypesResponse

scaleway_async.inference.v1.marshalling.unmarshal_Model(data: Any) → Model

scaleway_async.inference.v1.marshalling.unmarshal_ModelSupportInfo(data: Any) → ModelSupportInfo

scaleway_async.inference.v1.marshalling.unmarshal_ModelSupportedNode(data: Any) → ModelSupportedNode

scaleway_async.inference.v1.marshalling.unmarshal_ModelSupportedQuantization(data: Any) → ModelSupportedQuantization

scaleway_async.inference.v1.marshalling.unmarshal_NodeType(data: Any) → NodeType

scaleway_async.inference.v1.types module

class scaleway_async.inference.v1.types.CreateDeploymentRequest(model_id: 'str', node_type_name: 'str', endpoints: 'List[EndpointSpec]', region: 'Optional[ScwRegion]', name: 'Optional[str]', project_id: 'Optional[str]', accept_eula: 'Optional[bool]', tags: 'Optional[List[str]]', min_size: 'Optional[int]', max_size: 'Optional[int]', quantization: 'Optional[DeploymentQuantization]')

Bases: object

accept_eula: Optional[bool]

If the model has an EULA, you must accept it before proceeding.

The terms of the EULA can be retrieved using the GetModelEula API call.

endpoints: List[EndpointSpec]

List of endpoints to create.

max_size: Optional[int]

Defines the maximum size of the pool.

min_size: Optional[int]

Defines the minimum size of the pool.

model_id: str

ID of the model to use.

name: Optional[str]

Name of the deployment.

node_type_name: str

Name of the node type to use.

project_id: Optional[str]

ID of the Project to create the deployment in.

quantization: Optional[DeploymentQuantization]

Quantization settings to apply to this deployment.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

List of tags to apply to the deployment.

class scaleway_async.inference.v1.types.CreateEndpointRequest(deployment_id: 'str', endpoint: 'EndpointSpec', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to create the endpoint for.

endpoint: EndpointSpec

Specification of the endpoint.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.CreateModelRequest(source: 'ModelSource', region: 'Optional[ScwRegion]', name: 'Optional[str]', project_id: 'Optional[str]')

Bases: object

name: Optional[str]

Name of the model.

project_id: Optional[str]

ID of the Project to import the model in.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

source: ModelSource

Where to import the model from.

class scaleway_async.inference.v1.types.DeleteDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.DeleteEndpointRequest(endpoint_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

endpoint_id: str

ID of the endpoint to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.DeleteModelRequest(model_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

model_id: str

ID of the model to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.Deployment(id: 'str', name: 'str', project_id: 'str', status: 'DeploymentStatus', tags: 'List[str]', node_type_name: 'str', endpoints: 'List[Endpoint]', size: 'int', min_size: 'int', max_size: 'int', model_id: 'str', model_name: 'str', region: 'ScwRegion', error_message: 'Optional[str]', quantization: 'Optional[DeploymentQuantization]', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

created_at: Optional[datetime]

Creation date of the deployment.

endpoints: List[Endpoint]

List of endpoints.

error_message: Optional[str]

Displays information if your deployment is in error state.

id: str

Unique identifier.

max_size: int

Defines the maximum size of the pool.

min_size: int

Defines the minimum size of the pool.

model_id: str

ID of the model used for the deployment.

model_name: str

Name of the deployed model.

name: str

Name of the deployment.

node_type_name: str

Node type of the deployment.

project_id: str

Project ID.

quantization: Optional[DeploymentQuantization]

Quantization parameters for this deployment.

region: str

Region of the deployment.

size: int

Current size of the pool.

status: DeploymentStatus

Status of the deployment.

tags: List[str]

List of tags applied to the deployment.

updated_at: Optional[datetime]

Last modification date of the deployment.

class scaleway_async.inference.v1.types.DeploymentQuantization(bits: 'int')

Bases: object

bits: int

The number of bits each model parameter should be quantized to. The quantization method is chosen based on this value.

class scaleway_async.inference.v1.types.DeploymentStatus(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATING = 'creating'

DELETING = 'deleting'

DEPLOYING = 'deploying'

ERROR = 'error'

LOCKED = 'locked'

READY = 'ready'

UNKNOWN_STATUS = 'unknown_status'

class scaleway_async.inference.v1.types.Endpoint(id: 'str', url: 'str', disable_auth: 'bool', public_network: 'Optional[EndpointPublicNetworkDetails]', private_network: 'Optional[EndpointPrivateNetworkDetails]')

Bases: object

disable_auth: bool

Defines whether the authentication is disabled.

id: str

Unique identifier.

private_network: Optional[EndpointPrivateNetworkDetails]

public_network: Optional[EndpointPublicNetworkDetails]

url: str

For private endpoints, the URL will be accessible only from the Private Network.

In addition, private endpoints will expose a CA certificate that can be used to verify the server’s identity. This CA certificate can be retrieved using the GetDeploymentCertificate API call.

class scaleway_async.inference.v1.types.EndpointPrivateNetworkDetails(private_network_id: 'str')

Bases: object

private_network_id: str

class scaleway_async.inference.v1.types.EndpointPublicNetworkDetails

Bases: object

class scaleway_async.inference.v1.types.EndpointSpec(disable_auth: 'bool', public_network: 'Optional[EndpointPublicNetworkDetails]', private_network: 'Optional[EndpointPrivateNetworkDetails]')

Bases: object

disable_auth: bool

By default, deployments are protected by IAM authentication.

When setting this field to true, the authentication will be disabled.

private_network: Optional[EndpointPrivateNetworkDetails]

public_network: Optional[EndpointPublicNetworkDetails]

class scaleway_async.inference.v1.types.GetDeploymentCertificateRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.GetDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to get.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.GetModelRequest(model_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

model_id: str

ID of the model to get.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.ListDeploymentsRequest(region: 'Optional[ScwRegion]', page: 'Optional[int]', page_size: 'Optional[int]', order_by: 'Optional[ListDeploymentsRequestOrderBy]', project_id: 'Optional[str]', organization_id: 'Optional[str]', name: 'Optional[str]', tags: 'Optional[List[str]]')

Bases: object

name: Optional[str]

Filter by deployment name.

order_by: Optional[ListDeploymentsRequestOrderBy]

Order in which to return results.

organization_id: Optional[str]

Filter by Organization ID.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of deployments to return per page.

project_id: Optional[str]

Filter by Project ID.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

Filter by tags.

class scaleway_async.inference.v1.types.ListDeploymentsRequestOrderBy(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATED_AT_ASC = 'created_at_asc'

CREATED_AT_DESC = 'created_at_desc'

NAME_ASC = 'name_asc'

NAME_DESC = 'name_desc'

class scaleway_async.inference.v1.types.ListDeploymentsResponse(deployments: 'List[Deployment]', total_count: 'int')

Bases: object

deployments: List[Deployment]

List of deployments on the current page.

total_count: int

Total number of deployments.

class scaleway_async.inference.v1.types.ListModelsRequest(region: 'Optional[ScwRegion]', order_by: 'Optional[ListModelsRequestOrderBy]', page: 'Optional[int]', page_size: 'Optional[int]', project_id: 'Optional[str]', name: 'Optional[str]', tags: 'Optional[List[str]]')

Bases: object

name: Optional[str]

Filter by model name.

order_by: Optional[ListModelsRequestOrderBy]

Order in which to return results.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of models to return per page.

project_id: Optional[str]

Filter by Project ID.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

Filter by tags.

class scaleway_async.inference.v1.types.ListModelsRequestOrderBy(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATED_AT_ASC = 'created_at_asc'

CREATED_AT_DESC = 'created_at_desc'

DISPLAY_RANK_ASC = 'display_rank_asc'

NAME_ASC = 'name_asc'

NAME_DESC = 'name_desc'

class scaleway_async.inference.v1.types.ListModelsResponse(models: 'List[Model]', total_count: 'int')

Bases: object

models: List[Model]

List of models on the current page.

total_count: int

Total number of models.

class scaleway_async.inference.v1.types.ListNodeTypesRequest(include_disabled_types: 'bool', region: 'Optional[ScwRegion]', page: 'Optional[int]', page_size: 'Optional[int]')

Bases: object

include_disabled_types: bool

Include disabled node types in the response.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of node types to return per page.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.types.ListNodeTypesResponse(node_types: 'List[NodeType]', total_count: 'int')

Bases: object

node_types: List[NodeType]

List of node types.

total_count: int

Total number of node types.

class scaleway_async.inference.v1.types.Model(id: 'str', name: 'str', project_id: 'str', tags: 'List[str]', status: 'ModelStatus', description: 'str', has_eula: 'bool', region: 'ScwRegion', nodes_support: 'List[ModelSupportInfo]', parameter_size_bits: 'int', size_bytes: 'int', error_message: 'Optional[str]', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

created_at: Optional[datetime]

Creation date of the model.

description: str

Purpose of the model.

error_message: Optional[str]

Displays information if your model is in error state.

has_eula: bool

Defines whether the model has an end user license agreement.

id: str

Unique identifier.

name: str

Unique Name identifier.

nodes_support: List[ModelSupportInfo]

Supported nodes types with quantization options and context lengths.

parameter_size_bits: int

Size, in bits, of the model parameters.

project_id: str

Project ID.

region: str

Region of the model.

size_bytes: int

Total size, in bytes, of the model files.

status: ModelStatus

Status of the model.

tags: List[str]

List of tags applied to the model.

updated_at: Optional[datetime]

Last modification date of the model.

class scaleway_async.inference.v1.types.ModelSource(url: 'str', secret: 'Optional[str]')

Bases: object

secret: Optional[str]

url: str

class scaleway_async.inference.v1.types.ModelStatus(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

DOWNLOADING = 'downloading'

ERROR = 'error'

PREPARING = 'preparing'

READY = 'ready'

UNKNOWN_STATUS = 'unknown_status'

class scaleway_async.inference.v1.types.ModelSupportInfo(nodes: 'List[ModelSupportedNode]')

Bases: object

nodes: List[ModelSupportedNode]

List of supported node types.

class scaleway_async.inference.v1.types.ModelSupportedNode(node_type_name: 'str', quantizations: 'List[ModelSupportedQuantization]')

Bases: object

node_type_name: str

Supported node type.

quantizations: List[ModelSupportedQuantization]

Supported quantizations.

class scaleway_async.inference.v1.types.ModelSupportedQuantization(quantization_bits: 'int', allowed: 'bool', max_context_size: 'int')

Bases: object

allowed: bool

Tells whether this quantization is allowed for this node type.

max_context_size: int

Maximum inference context size available for this node type and quantization.

quantization_bits: int

Number of bits for this supported quantization.

class scaleway_async.inference.v1.types.NodeType(name: 'str', stock_status: 'NodeTypeStock', description: 'str', vcpus: 'int', memory: 'int', vram: 'int', disabled: 'bool', beta: 'bool', gpus: 'int', region: 'ScwRegion', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

beta: bool

The node type is currently in beta.

created_at: Optional[datetime]

Creation date of the node type.

description: str

Current specs of the offer.

disabled: bool

The node type is currently disabled.

gpus: int

Number of GPUs.

memory: int

Quantity of RAM.

name: str

Name of the node type.

region: str

Region of the node type.

stock_status: NodeTypeStock

Current stock status for the node type.

updated_at: Optional[datetime]

Last modification date of the node type.

vcpus: int

Number of virtual CPUs.

vram: int

Quantity of GPU RAM.

class scaleway_async.inference.v1.types.NodeTypeStock(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

AVAILABLE = 'available'

LOW_STOCK = 'low_stock'

OUT_OF_STOCK = 'out_of_stock'

UNKNOWN_STOCK = 'unknown_stock'

class scaleway_async.inference.v1.types.UpdateDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]', name: 'Optional[str]', tags: 'Optional[List[str]]', min_size: 'Optional[int]', max_size: 'Optional[int]', model_id: 'Optional[str]', quantization: 'Optional[DeploymentQuantization]')

Bases: object

deployment_id: str

ID of the deployment to update.

max_size: Optional[int]

Defines the new maximum size of the pool.

min_size: Optional[int]

Defines the new minimum size of the pool.

model_id: Optional[str]

Id of the model to set to the deployment.

name: Optional[str]

Name of the deployment.

quantization: Optional[DeploymentQuantization]

Quantization to use to the deployment.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

List of tags to apply to the deployment.

class scaleway_async.inference.v1.types.UpdateEndpointRequest(endpoint_id: 'str', region: 'Optional[ScwRegion]', disable_auth: 'Optional[bool]')

Bases: object

disable_auth: Optional[bool]

By default, deployments are protected by IAM authentication.

When setting this field to true, the authentication will be disabled.

endpoint_id: str

ID of the endpoint to update.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

Module contents

class scaleway_async.inference.v1.CreateDeploymentRequest(model_id: 'str', node_type_name: 'str', endpoints: 'List[EndpointSpec]', region: 'Optional[ScwRegion]', name: 'Optional[str]', project_id: 'Optional[str]', accept_eula: 'Optional[bool]', tags: 'Optional[List[str]]', min_size: 'Optional[int]', max_size: 'Optional[int]', quantization: 'Optional[DeploymentQuantization]')

Bases: object

accept_eula: Optional[bool]

If the model has an EULA, you must accept it before proceeding.

The terms of the EULA can be retrieved using the GetModelEula API call.

endpoints: List[EndpointSpec]

List of endpoints to create.

max_size: Optional[int]

Defines the maximum size of the pool.

min_size: Optional[int]

Defines the minimum size of the pool.

model_id: str

ID of the model to use.

name: Optional[str]

Name of the deployment.

node_type_name: str

Name of the node type to use.

project_id: Optional[str]

ID of the Project to create the deployment in.

quantization: Optional[DeploymentQuantization]

Quantization settings to apply to this deployment.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

List of tags to apply to the deployment.

class scaleway_async.inference.v1.CreateEndpointRequest(deployment_id: 'str', endpoint: 'EndpointSpec', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to create the endpoint for.

endpoint: EndpointSpec

Specification of the endpoint.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.CreateModelRequest(source: 'ModelSource', region: 'Optional[ScwRegion]', name: 'Optional[str]', project_id: 'Optional[str]')

Bases: object

name: Optional[str]

Name of the model.

project_id: Optional[str]

ID of the Project to import the model in.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

source: ModelSource

Where to import the model from.

class scaleway_async.inference.v1.DeleteDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.DeleteEndpointRequest(endpoint_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

endpoint_id: str

ID of the endpoint to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.DeleteModelRequest(model_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

model_id: str

ID of the model to delete.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.Deployment(id: 'str', name: 'str', project_id: 'str', status: 'DeploymentStatus', tags: 'List[str]', node_type_name: 'str', endpoints: 'List[Endpoint]', size: 'int', min_size: 'int', max_size: 'int', model_id: 'str', model_name: 'str', region: 'ScwRegion', error_message: 'Optional[str]', quantization: 'Optional[DeploymentQuantization]', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

created_at: Optional[datetime]

Creation date of the deployment.

endpoints: List[Endpoint]

List of endpoints.

error_message: Optional[str]

Displays information if your deployment is in error state.

id: str

Unique identifier.

max_size: int

Defines the maximum size of the pool.

min_size: int

Defines the minimum size of the pool.

model_id: str

ID of the model used for the deployment.

model_name: str

Name of the deployed model.

name: str

Name of the deployment.

node_type_name: str

Node type of the deployment.

project_id: str

Project ID.

quantization: Optional[DeploymentQuantization]

Quantization parameters for this deployment.

region: str

Region of the deployment.

size: int

Current size of the pool.

status: DeploymentStatus

Status of the deployment.

tags: List[str]

List of tags applied to the deployment.

updated_at: Optional[datetime]

Last modification date of the deployment.

class scaleway_async.inference.v1.DeploymentQuantization(bits: 'int')

Bases: object

bits: int

The number of bits each model parameter should be quantized to. The quantization method is chosen based on this value.

class scaleway_async.inference.v1.DeploymentStatus(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATING = 'creating'

DELETING = 'deleting'

DEPLOYING = 'deploying'

ERROR = 'error'

LOCKED = 'locked'

READY = 'ready'

UNKNOWN_STATUS = 'unknown_status'

class scaleway_async.inference.v1.Endpoint(id: 'str', url: 'str', disable_auth: 'bool', public_network: 'Optional[EndpointPublicNetworkDetails]', private_network: 'Optional[EndpointPrivateNetworkDetails]')

Bases: object

disable_auth: bool

Defines whether the authentication is disabled.

id: str

Unique identifier.

private_network: Optional[EndpointPrivateNetworkDetails]

public_network: Optional[EndpointPublicNetworkDetails]

url: str

For private endpoints, the URL will be accessible only from the Private Network.

In addition, private endpoints will expose a CA certificate that can be used to verify the server’s identity. This CA certificate can be retrieved using the GetDeploymentCertificate API call.

class scaleway_async.inference.v1.EndpointPrivateNetworkDetails(private_network_id: 'str')

Bases: object

private_network_id: str

class scaleway_async.inference.v1.EndpointPublicNetworkDetails

Bases: object

class scaleway_async.inference.v1.EndpointSpec(disable_auth: 'bool', public_network: 'Optional[EndpointPublicNetworkDetails]', private_network: 'Optional[EndpointPrivateNetworkDetails]')

Bases: object

disable_auth: bool

By default, deployments are protected by IAM authentication.

When setting this field to true, the authentication will be disabled.

private_network: Optional[EndpointPrivateNetworkDetails]

public_network: Optional[EndpointPublicNetworkDetails]

class scaleway_async.inference.v1.GetDeploymentCertificateRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.GetDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

deployment_id: str

ID of the deployment to get.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.GetModelRequest(model_id: 'str', region: 'Optional[ScwRegion]')

Bases: object

model_id: str

ID of the model to get.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.InferenceV1API(client: Client, *, bypass_validation: bool = False)

Bases: API

This API allows you to handle your Managed Inference services.

async create_deployment(*, model_id: str, node_type_name: str, endpoints: List[EndpointSpec], region: Optional[str] = None, name: Optional[str] = None, project_id: Optional[str] = None, accept_eula: Optional[bool] = None, tags: Optional[List[str]] = None, min_size: Optional[int] = None, max_size: Optional[int] = None, quantization: Optional[DeploymentQuantization] = None) → Deployment

Create a deployment. Create a new inference deployment related to a specific model. :param model_id: ID of the model to use. :param node_type_name: Name of the node type to use. :param endpoints: List of endpoints to create. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the deployment. :param project_id: ID of the Project to create the deployment in. :param accept_eula: If the model has an EULA, you must accept it before proceeding. The terms of the EULA can be retrieved using the GetModelEula API call. :param tags: List of tags to apply to the deployment. :param min_size: Defines the minimum size of the pool. :param max_size: Defines the maximum size of the pool. :param quantization: Quantization settings to apply to this deployment. :return: Deployment

Usage:

result = await api.create_deployment(
    model_id="example",
    node_type_name="example",
    endpoints=[],
)

async create_endpoint(*, deployment_id: str, endpoint: EndpointSpec, region: Optional[str] = None) → Endpoint

Create an endpoint. Create a new Endpoint related to a specific deployment. :param deployment_id: ID of the deployment to create the endpoint for. :param endpoint: Specification of the endpoint. :param region: Region to target. If none is passed will use default region from the config. :return: Endpoint

Usage:

result = await api.create_endpoint(
    deployment_id="example",
    endpoint=EndpointSpec(),
)

async create_model(*, source: ModelSource, region: Optional[str] = None, name: Optional[str] = None, project_id: Optional[str] = None) → Model

Import a model. Import a new model to your model library. :param source: Where to import the model from. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the model. :param project_id: ID of the Project to import the model in. :return: Model

Usage:

result = await api.create_model(
    source=ModelSource(),
)

async delete_deployment(*, deployment_id: str, region: Optional[str] = None) → Deployment

Delete a deployment. Delete an existing inference deployment. :param deployment_id: ID of the deployment to delete. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.delete_deployment(
    deployment_id="example",
)

async delete_endpoint(*, endpoint_id: str, region: Optional[str] = None) → None

Delete an endpoint. Delete an existing Endpoint. :param endpoint_id: ID of the endpoint to delete. :param region: Region to target. If none is passed will use default region from the config.

Usage:

result = await api.delete_endpoint(
    endpoint_id="example",
)

async delete_model(*, model_id: str, region: Optional[str] = None) → None

Delete a model. Delete an existing model from your model library. :param model_id: ID of the model to delete. :param region: Region to target. If none is passed will use default region from the config.

Usage:

result = await api.delete_model(
    model_id="example",
)

async get_deployment(*, deployment_id: str, region: Optional[str] = None) → Deployment

Get a deployment. Get the deployment for the given ID. :param deployment_id: ID of the deployment to get. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.get_deployment(
    deployment_id="example",
)

async get_deployment_certificate(*, deployment_id: str, region: Optional[str] = None) → ScwFile

Get the CA certificate. Get the CA certificate used for the deployment of private endpoints. The CA certificate will be returned as a PEM file. :param deployment_id: :param region: Region to target. If none is passed will use default region from the config. :return: ScwFile

Usage:

result = await api.get_deployment_certificate(
    deployment_id="example",
)

async get_model(*, model_id: str, region: Optional[str] = None) → Model

Get a model. Get the model for the given ID. :param model_id: ID of the model to get. :param region: Region to target. If none is passed will use default region from the config. :return: Model

Usage:

result = await api.get_model(
    model_id="example",
)

async list_deployments(*, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None, order_by: Optional[ListDeploymentsRequestOrderBy] = None, project_id: Optional[str] = None, organization_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → ListDeploymentsResponse

List inference deployments. List all your inference deployments. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of deployments to return per page. :param order_by: Order in which to return results. :param project_id: Filter by Project ID. :param organization_id: Filter by Organization ID. :param name: Filter by deployment name. :param tags: Filter by tags. :return: ListDeploymentsResponse

Usage:

result = await api.list_deployments()

async list_deployments_all(*, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None, order_by: Optional[ListDeploymentsRequestOrderBy] = None, project_id: Optional[str] = None, organization_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → List[Deployment]

List inference deployments. List all your inference deployments. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of deployments to return per page. :param order_by: Order in which to return results. :param project_id: Filter by Project ID. :param organization_id: Filter by Organization ID. :param name: Filter by deployment name. :param tags: Filter by tags. :return: List[Deployment]

Usage:

result = await api.list_deployments_all()

async list_models(*, region: Optional[str] = None, order_by: Optional[ListModelsRequestOrderBy] = None, page: Optional[int] = None, page_size: Optional[int] = None, project_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → ListModelsResponse

List models. List all available models. :param region: Region to target. If none is passed will use default region from the config. :param order_by: Order in which to return results. :param page: Page number to return. :param page_size: Maximum number of models to return per page. :param project_id: Filter by Project ID. :param name: Filter by model name. :param tags: Filter by tags. :return: ListModelsResponse

Usage:

result = await api.list_models()

async list_models_all(*, region: Optional[str] = None, order_by: Optional[ListModelsRequestOrderBy] = None, page: Optional[int] = None, page_size: Optional[int] = None, project_id: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None) → List[Model]

List models. List all available models. :param region: Region to target. If none is passed will use default region from the config. :param order_by: Order in which to return results. :param page: Page number to return. :param page_size: Maximum number of models to return per page. :param project_id: Filter by Project ID. :param name: Filter by model name. :param tags: Filter by tags. :return: List[Model]

Usage:

result = await api.list_models_all()

async list_node_types(*, include_disabled_types: bool, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None) → ListNodeTypesResponse

List available node types. List all available node types. By default, the node types returned in the list are ordered by creation date in ascending order, though this can be modified via the order_by field. :param include_disabled_types: Include disabled node types in the response. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of node types to return per page. :return: ListNodeTypesResponse

Usage:

result = await api.list_node_types(
    include_disabled_types=False,
)

async list_node_types_all(*, include_disabled_types: bool, region: Optional[str] = None, page: Optional[int] = None, page_size: Optional[int] = None) → List[NodeType]

List available node types. List all available node types. By default, the node types returned in the list are ordered by creation date in ascending order, though this can be modified via the order_by field. :param include_disabled_types: Include disabled node types in the response. :param region: Region to target. If none is passed will use default region from the config. :param page: Page number to return. :param page_size: Maximum number of node types to return per page. :return: List[NodeType]

Usage:

result = await api.list_node_types_all(
    include_disabled_types=False,
)

async update_deployment(*, deployment_id: str, region: Optional[str] = None, name: Optional[str] = None, tags: Optional[List[str]] = None, min_size: Optional[int] = None, max_size: Optional[int] = None, model_id: Optional[str] = None, quantization: Optional[DeploymentQuantization] = None) → Deployment

Update a deployment. Update an existing inference deployment. :param deployment_id: ID of the deployment to update. :param region: Region to target. If none is passed will use default region from the config. :param name: Name of the deployment. :param tags: List of tags to apply to the deployment. :param min_size: Defines the new minimum size of the pool. :param max_size: Defines the new maximum size of the pool. :param model_id: Id of the model to set to the deployment. :param quantization: Quantization to use to the deployment. :return: Deployment

Usage:

result = await api.update_deployment(
    deployment_id="example",
)

async update_endpoint(*, endpoint_id: str, region: Optional[str] = None, disable_auth: Optional[bool] = None) → Endpoint

Update an endpoint. Update an existing Endpoint. :param endpoint_id: ID of the endpoint to update. :param region: Region to target. If none is passed will use default region from the config. :param disable_auth: By default, deployments are protected by IAM authentication. When setting this field to true, the authentication will be disabled. :return: Endpoint

Usage:

result = await api.update_endpoint(
    endpoint_id="example",
)

async wait_for_deployment(*, deployment_id: str, region: Optional[str] = None, options: Optional[WaitForOptions[Deployment, Union[bool, Awaitable[bool]]]] = None) → Deployment

Get a deployment. Get the deployment for the given ID. :param deployment_id: ID of the deployment to get. :param region: Region to target. If none is passed will use default region from the config. :return: Deployment

Usage:

result = await api.get_deployment(
    deployment_id="example",
)

async wait_for_model(*, model_id: str, region: Optional[str] = None, options: Optional[WaitForOptions[Model, Union[bool, Awaitable[bool]]]] = None) → Model

Get a model. Get the model for the given ID. :param model_id: ID of the model to get. :param region: Region to target. If none is passed will use default region from the config. :return: Model

Usage:

result = await api.get_model(
    model_id="example",
)

class scaleway_async.inference.v1.ListDeploymentsRequest(region: 'Optional[ScwRegion]', page: 'Optional[int]', page_size: 'Optional[int]', order_by: 'Optional[ListDeploymentsRequestOrderBy]', project_id: 'Optional[str]', organization_id: 'Optional[str]', name: 'Optional[str]', tags: 'Optional[List[str]]')

Bases: object

name: Optional[str]

Filter by deployment name.

order_by: Optional[ListDeploymentsRequestOrderBy]

Order in which to return results.

organization_id: Optional[str]

Filter by Organization ID.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of deployments to return per page.

project_id: Optional[str]

Filter by Project ID.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

Filter by tags.

class scaleway_async.inference.v1.ListDeploymentsRequestOrderBy(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATED_AT_ASC = 'created_at_asc'

CREATED_AT_DESC = 'created_at_desc'

NAME_ASC = 'name_asc'

NAME_DESC = 'name_desc'

class scaleway_async.inference.v1.ListDeploymentsResponse(deployments: 'List[Deployment]', total_count: 'int')

Bases: object

deployments: List[Deployment]

List of deployments on the current page.

total_count: int

Total number of deployments.

class scaleway_async.inference.v1.ListModelsRequest(region: 'Optional[ScwRegion]', order_by: 'Optional[ListModelsRequestOrderBy]', page: 'Optional[int]', page_size: 'Optional[int]', project_id: 'Optional[str]', name: 'Optional[str]', tags: 'Optional[List[str]]')

Bases: object

name: Optional[str]

Filter by model name.

order_by: Optional[ListModelsRequestOrderBy]

Order in which to return results.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of models to return per page.

project_id: Optional[str]

Filter by Project ID.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

Filter by tags.

class scaleway_async.inference.v1.ListModelsRequestOrderBy(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

CREATED_AT_ASC = 'created_at_asc'

CREATED_AT_DESC = 'created_at_desc'

DISPLAY_RANK_ASC = 'display_rank_asc'

NAME_ASC = 'name_asc'

NAME_DESC = 'name_desc'

class scaleway_async.inference.v1.ListModelsResponse(models: 'List[Model]', total_count: 'int')

Bases: object

models: List[Model]

List of models on the current page.

total_count: int

Total number of models.

class scaleway_async.inference.v1.ListNodeTypesRequest(include_disabled_types: 'bool', region: 'Optional[ScwRegion]', page: 'Optional[int]', page_size: 'Optional[int]')

Bases: object

include_disabled_types: bool

Include disabled node types in the response.

page: Optional[int]

Page number to return.

page_size: Optional[int]

Maximum number of node types to return per page.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

class scaleway_async.inference.v1.ListNodeTypesResponse(node_types: 'List[NodeType]', total_count: 'int')

Bases: object

node_types: List[NodeType]

List of node types.

total_count: int

Total number of node types.

class scaleway_async.inference.v1.Model(id: 'str', name: 'str', project_id: 'str', tags: 'List[str]', status: 'ModelStatus', description: 'str', has_eula: 'bool', region: 'ScwRegion', nodes_support: 'List[ModelSupportInfo]', parameter_size_bits: 'int', size_bytes: 'int', error_message: 'Optional[str]', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

created_at: Optional[datetime]

Creation date of the model.

description: str

Purpose of the model.

error_message: Optional[str]

Displays information if your model is in error state.

has_eula: bool

Defines whether the model has an end user license agreement.

id: str

Unique identifier.

name: str

Unique Name identifier.

nodes_support: List[ModelSupportInfo]

Supported nodes types with quantization options and context lengths.

parameter_size_bits: int

Size, in bits, of the model parameters.

project_id: str

Project ID.

region: str

Region of the model.

size_bytes: int

Total size, in bytes, of the model files.

status: ModelStatus

Status of the model.

tags: List[str]

List of tags applied to the model.

updated_at: Optional[datetime]

Last modification date of the model.

class scaleway_async.inference.v1.ModelSource(url: 'str', secret: 'Optional[str]')

Bases: object

secret: Optional[str]

url: str

class scaleway_async.inference.v1.ModelStatus(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

DOWNLOADING = 'downloading'

ERROR = 'error'

PREPARING = 'preparing'

READY = 'ready'

UNKNOWN_STATUS = 'unknown_status'

class scaleway_async.inference.v1.ModelSupportInfo(nodes: 'List[ModelSupportedNode]')

Bases: object

nodes: List[ModelSupportedNode]

List of supported node types.

class scaleway_async.inference.v1.ModelSupportedNode(node_type_name: 'str', quantizations: 'List[ModelSupportedQuantization]')

Bases: object

node_type_name: str

Supported node type.

quantizations: List[ModelSupportedQuantization]

Supported quantizations.

class scaleway_async.inference.v1.ModelSupportedQuantization(quantization_bits: 'int', allowed: 'bool', max_context_size: 'int')

Bases: object

allowed: bool

Tells whether this quantization is allowed for this node type.

max_context_size: int

Maximum inference context size available for this node type and quantization.

quantization_bits: int

Number of bits for this supported quantization.

class scaleway_async.inference.v1.NodeType(name: 'str', stock_status: 'NodeTypeStock', description: 'str', vcpus: 'int', memory: 'int', vram: 'int', disabled: 'bool', beta: 'bool', gpus: 'int', region: 'ScwRegion', created_at: 'Optional[datetime]', updated_at: 'Optional[datetime]')

Bases: object

beta: bool

The node type is currently in beta.

created_at: Optional[datetime]

Creation date of the node type.

description: str

Current specs of the offer.

disabled: bool

The node type is currently disabled.

gpus: int

Number of GPUs.

memory: int

Quantity of RAM.

name: str

Name of the node type.

region: str

Region of the node type.

stock_status: NodeTypeStock

Current stock status for the node type.

updated_at: Optional[datetime]

Last modification date of the node type.

vcpus: int

Number of virtual CPUs.

vram: int

Quantity of GPU RAM.

class scaleway_async.inference.v1.NodeTypeStock(value: str, names: Optional[Any] = None, *args: Any, **kwargs: Any)

Bases: str, Enum

An enumeration.

AVAILABLE = 'available'

LOW_STOCK = 'low_stock'

OUT_OF_STOCK = 'out_of_stock'

UNKNOWN_STOCK = 'unknown_stock'

class scaleway_async.inference.v1.UpdateDeploymentRequest(deployment_id: 'str', region: 'Optional[ScwRegion]', name: 'Optional[str]', tags: 'Optional[List[str]]', min_size: 'Optional[int]', max_size: 'Optional[int]', model_id: 'Optional[str]', quantization: 'Optional[DeploymentQuantization]')

Bases: object

deployment_id: str

ID of the deployment to update.

max_size: Optional[int]

Defines the new maximum size of the pool.

min_size: Optional[int]

Defines the new minimum size of the pool.

model_id: Optional[str]

Id of the model to set to the deployment.

name: Optional[str]

Name of the deployment.

quantization: Optional[DeploymentQuantization]

Quantization to use to the deployment.

region: Optional[str]

Region to target. If none is passed will use default region from the config.

tags: Optional[List[str]]

List of tags to apply to the deployment.

class scaleway_async.inference.v1.UpdateEndpointRequest(endpoint_id: 'str', region: 'Optional[ScwRegion]', disable_auth: 'Optional[bool]')

Bases: object

disable_auth: Optional[bool]

By default, deployments are protected by IAM authentication.

When setting this field to true, the authentication will be disabled.

endpoint_id: str

ID of the endpoint to update.

region: Optional[str]

Region to target. If none is passed will use default region from the config.