Inference

Last verified 12 Oct 2026

Inference provides a single control plane for managing inference workflows. It includes a Model Catalog where you can view available foundation models, including both DigitalOcean-hosted and third-party commercial models, compare model capabilities and pricing, use routing to match inference requests to the best-fit model, and run inference using serverless or dedicated deployments.

Browse Models in Model Catalog

Identify the right model for your use case by filtering available foundation models by capabilities and price.

Use Model Playground

Test and compare foundation models in the Model Playground.

Use Serverless Inference

Send API requests directly to foundation models without creating an AI agent or managing infrastructure.

Deploy to Dedicated Inference Endpoints

Deploy open-source and commercial LLMs on dedicated GPUs as an inference endpoint.

Use Inference Router

Send requests to inference-router and let it pick the best model for each request.

Evaluate Models

Determine which model best fits your specific use case.

Use Batch Inference

Batch Inference lets you run large collections of LLM requests as a single asynchronous job.

Use Agent Platform

Use to build fully-managed AI agents with knowledge bases for retrieval-augmented generation, multi-agent routing, guardrails, and more.

Latest Updates

12 October 2026

  • Inference Router has been redesigned as a single managed router. Send requests to the inference-router model name, optionally with a routing_preference of balanced, budget, or quality, and the router selects the best model for each request.

    Preset routers and the Create Router flow have been removed from the Control Panel. Existing custom routers and preset router slugs continue to serve requests. For more information, see Use Inference Router.

9 October 2026

8 October 2026

  • DigitalOcean Signals is now available in public preview for Harness Runtime agents and Serverless Inference. Signals detects behavioral signals in conversations, such as a user asking for a human, a failed tool call, or an execution loop, so you can find the conversations that went wrong without reading every transcript. Signals collects data only after you opt in. See Use Signals with Harness Runtime and Use Signals with Serverless Inference.

  • Prompt caching for DigitalOcean-hosted models is now in general availability. All DigitalOcean-hosted models cache context automatically, with no code changes and no cache-write charge. Cache affinity routes requests with matching prefixes to the replica that holds the cache. You can isolate cached context within your account with the optional prompt_cache_key key, and view your cache hit rate in the control panel. For more information, see Use Prompt Caching.

For more information, see the full release notes.

We can't find any results for your search.

Try using different keywords or simplifying your search terms.