For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Usage

client.dedicated_inferences.patch(
    dedicated_inference_id="6b5c619c-359c-44ca-87e2-47e98170c01d",
    body={
        "spec": {...},
        "access_tokens": {...},
    },
)

Description

Update an existing Dedicated Inference. Send a PATCH request to /v2/dedicated-inferences/{dedicated_inference_id} with updated spec and/or access_tokens. Status will move to updating and return to active when done.

Parameters

dedicated_inference_id string required

A unique identifier for a Dedicated Inference instance.

spec object optional

Structured configuration for a Dedicated Inference deployment.

Show child properties

version integer required

Example: 1

Spec version.

name string required

Example: new-dedicated-inference

Name of the Dedicated Inference. Must be unique within the team.

Max length: 255

region string required

DigitalOcean region where the Dedicated Inference is hosted.

One of: atl1, nyc2, tor1

vpc object required

Show child properties

uuid string required

Example: 997615ce-132d-4bae-9270-9ee21b395e5d

VPC UUID for the Dedicated Inference.

enable_public_endpoint boolean required

Whether to expose a public LLM endpoint.

model_deployments array of objects required

At least one model deployment is required.

Show child properties

model_id string optional

Used to identify an existing deployment when updating; empty means create new.

model_slug string optional

Example: mistral/mistral-7b-instruct-v3

Model identifier (e.g. Hugging Face slug).

model_provider string optional

Example: hugging_face

Model provider.

workload_config object optional

Workload-specific configuration (e.g. ISL/OSL in future).

accelerators array of objects optional

Accelerator configuration for this deployment.

Show child properties

scale integer required

Example: 1

Number of accelerator instances.

Min: 1

type string required

Example: prefill_decode

Accelerator type (e.g. prefill_decode).

accelerator_slug string required

Example: gpu-mi300x1-192gb

DigitalOcean GPU slug.

status string optional read-only

Current state of the Accelerator.

One of: new, provisioning, active

access_tokens object optional

Provider tokens for model access (e.g. gated Hugging Face models).

Show child properties

hugging_face_token string optional

Example: $HF_TOKEN

Hugging Face token required for gated models.

Request Sample

import os
from pydo import Client

client = Client(token=os.environ.get("DIGITALOCEAN_TOKEN"))

req = {
  "spec": {
    "version": 1,
    "name": "new-dedicated-inference",
    "region": "atl1",
    "vpc": {
      "uuid": "997615ce-132d-4bae-9270-9ee21b395e5d"
    },
    "enable_public_endpoint": True,
    "model_deployments": []
  },
  "access_tokens": {
    "hugging_face_token": "$HF_TOKEN"
  }
}

resp = client.dedicated_inferences.patch(dedicated_inference_id="6b5c619c-359c-44ca-87e2-47e98170c01d", body=req)

Response Example

{
  "dedicated_inference": {
    "id": "6b5c619c-359c-44ca-87e2-47e98170c01d",
    "status": "active",
    "region": "atl1",
    "vpc_uuid": "997615ce-132d-4bae-9270-9ee21b395e5d",
    "spec": {
      "version": 1,
      "name": "new-dedicated-inference",
      "region": "atl1",
      "enable_public_endpoint": true,
      "model_deployments": []
    },
    "pending_deployment_spec": {
      "id": "7c6d729d-360d-44db-88e3-58e98281d12e",
      "version": 1,
      "name": "new-dedicated-inference",
      "status": "provisioning",
      "enable_public_endpoint": true,
      "model_deployments": [],
      "created_at": "2024-01-09T20:44:32Z",
      "updated_at": "2024-01-09T20:44:32Z"
    },
    "endpoints": {
      "public_endpoint_fqdn": "https://b4bfug4jc41kts2ro54if91eo-public-dedicated-inference.do-infra.ai",
      "private_endpoint_fqdn": "https://b4bfug4jc41kts2ro54if91eo-private-dedicated-inference.do-infra.ai"
    },
    "created_at": "2024-01-09T20:44:32Z",
    "updated_at": "2024-01-09T20:44:32Z"
  }
}

More Information

See /v2/dedicated-inferences/{dedicated_inference_id} in the API reference for additional detail on responses, headers, parameters, and more.