For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Usage

client.dedicated_inferences.create(
    body={
        "spec": {...},
        "access_tokens": {...},
    },
)

Description

Create a new Dedicated Inference for your team. Send a POST request to /v2/dedicated-inferences with a spec object (version, name, region, vpc, enable_public_endpoint, model_deployments) and optional access_tokens (e.g. hugging_face_token for gated models). The response code 202 Accepted indicates the request was accepted for processing; it does not indicate success or failure. The token value is returned only on create; store it securely.

Parameters

spec object required

Structured configuration for a Dedicated Inference deployment.

Show child properties

version integer required

Example: 1

Spec version.

name string required

Example: new-dedicated-inference

Name of the Dedicated Inference. Must be unique within the team.

Max length: 255

region string required

DigitalOcean region where the Dedicated Inference is hosted.

One of: atl1, nyc2, tor1

vpc object required

Show child properties

uuid string required

Example: 997615ce-132d-4bae-9270-9ee21b395e5d

VPC UUID for the Dedicated Inference.

enable_public_endpoint boolean required

Whether to expose a public LLM endpoint.

model_deployments array of objects required

At least one model deployment is required.

Show child properties

model_id string optional

Used to identify an existing deployment when updating; empty means create new.

model_slug string optional

Example: mistral/mistral-7b-instruct-v3

Model identifier (e.g. Hugging Face slug).

model_provider string optional

Example: hugging_face

Model provider.

workload_config object optional

Workload-specific configuration (e.g. ISL/OSL in future).

accelerators array of objects optional

Accelerator configuration for this deployment.

Show child properties

scale integer required

Example: 1

Number of accelerator instances.

Min: 1

type string required

Example: prefill_decode

Accelerator type (e.g. prefill_decode).

accelerator_slug string required

Example: gpu-mi300x1-192gb

DigitalOcean GPU slug.

status string optional read-only

Current state of the Accelerator.

One of: new, provisioning, active

access_tokens object optional

Example: {'hugging_face_token': '$HF_TOKEN'}

Key-value pairs for provider tokens (e.g. Hugging Face).

Request Sample

import os
from pydo import Client

client = Client(token=os.environ.get("DIGITALOCEAN_TOKEN"))

req = {
  "spec": {
    "version": 1,
    "name": "new-dedicated-inference",
    "region": "atl1",
    "vpc": {
      "uuid": "997615ce-132d-4bae-9270-9ee21b395e5d"
    },
    "enable_public_endpoint": True,
    "model_deployments": []
  },
  "access_tokens": {
    "hugging_face_token": "$HF_TOKEN"
  }
}

resp = client.dedicated_inferences.create(body=req)

Response Example

{
  "dedicated_inference": {
    "id": "6b5c619c-359c-44ca-87e2-47e98170c01d",
    "status": "active",
    "region": "atl1",
    "vpc_uuid": "997615ce-132d-4bae-9270-9ee21b395e5d",
    "spec": {
      "version": 1,
      "name": "new-dedicated-inference",
      "region": "atl1",
      "enable_public_endpoint": true,
      "model_deployments": []
    },
    "pending_deployment_spec": {
      "id": "7c6d729d-360d-44db-88e3-58e98281d12e",
      "version": 1,
      "name": "new-dedicated-inference",
      "status": "provisioning",
      "enable_public_endpoint": true,
      "model_deployments": [],
      "created_at": "2024-01-09T20:44:32Z",
      "updated_at": "2024-01-09T20:44:32Z"
    },
    "endpoints": {
      "public_endpoint_fqdn": "https://b4bfug4jc41kts2ro54if91eo-public-dedicated-inference.do-infra.ai",
      "private_endpoint_fqdn": "https://b4bfug4jc41kts2ro54if91eo-private-dedicated-inference.do-infra.ai"
    },
    "created_at": "2024-01-09T20:44:32Z",
    "updated_at": "2024-01-09T20:44:32Z"
  },
  "token": {
    "id": "01333f14-a903-4b8e-92b3-363a767aa052",
    "name": "first-token",
    "value": "di_xxxxxxxxxxxxxxxxxxxxxxxx",
    "created_at": "2024-01-09T20:44:32Z",
    "is_managed": false
  }
}

More Information

See /v2/dedicated-inferences in the API reference for additional detail on responses, headers, parameters, and more.