Inference Release Notes

Last verified 8 May 2026

Copy page as Markdown View page as Markdown

October 2026

12 October

  • Inference Router has been redesigned as a single managed router. Send requests to the inference-router model name, optionally with a routing_preference of balanced, budget, or quality, and the router selects the best model for each request.

    Preset routers and the Create Router flow have been removed from the Control Panel. Existing custom routers and preset router slugs continue to serve requests. For more information, see Use Inference Router.

9 October

8 October

  • Prompt caching for DigitalOcean-hosted models is now in general availability. All DigitalOcean-hosted models cache context automatically, with no code changes and no cache-write charge. Cache affinity routes requests with matching prefixes to the replica that holds the cache. You can isolate cached context within your account with the optional prompt_cache_key key, and view your cache hit rate in the control panel. For more information, see Use Prompt Caching.

  • DigitalOcean Signals is now available in public preview for Harness Runtime agents and Serverless Inference. Signals detects behavioral signals in conversations, such as a user asking for a human, a failed tool call, or an execution loop, so you can find the conversations that went wrong without reading every transcript. Signals collects data only after you opt in. See Use Signals with Harness Runtime and Use Signals with Serverless Inference.

7 October

  • The Cost Analysis tab of the Billing page now includes Serverless Inference Live Usage, which lists the estimated cost and token counts of your Serverless Inference requests by model and model access key, for both teams and organizations. Data refreshes every 10 minutes and covers the last 30 days. For more information, see View Serverless Inference Live Usage.

1 October

  • The Agent Development Kit (ADK), including the gradient-adk Python package and the gradient CLI, is deprecated. You cannot deploy any new agents with ADK. Agents already deployed with ADK continue to run normally and you are billed for model usage. If you are using a DigitalOcean-hosted model, you are charged for those model keys. Migration guidance and end-of-support timeline will be published in the future.

  • You can now prepay for Serverless Inference and Managed Agents with the Inference and Agents balance, which no other products draw from. Add $5 to $500 at a time or turn on auto-reload from the Billing page. Funds do not expire. You can also pay for eligible usage at a discount with a monthly plan, now in public preview. See Inference and Agents Balance and How to Pay for Harness Runtime.

September 2026

30 September

29 September

  • Released v1.175.0 of doctl, the official DigitalOcean CLI. This release deprecates the doctl gradient commands.

    doctl gradient agent commands have no CLI replacement. To manage agents, use the control panel or API.

    doctl gradient knowledge-base commands work but will be removed in a future release. Use doctl knowledge-base instead.

    To list models, regions, and OpenAI keys, use the doctl serverless-inference commands.

28 September

25 September

23 September

  • The following TypeSafe AI model is now available on DigitalOcean Inference for inference:

    For more information, see the Available Models page.

22 September

14 September

8 September

4 September

1 September

August 2026

28 August

27 August

20 August

  • Qwen3.8-2.4T-A95B model available on DigitalOcean Inference has been upgraded to Qwen3.8-Max. Qwen3.8-Max supports vision and video input and a 1M-token context window. For more information, see the Available Models page.

19 August

  • Inference Router now uses cache-aware routing to maximize prompt cache reuse and automatically applies prompt caching to eligible Anthropic requests. You can opt out of prompt caching for supported models with X-Model-Affinity: none and control cache-aware model switching with the x-routing-max-switch-spend-pct header. For more information, see Use Prompt Caching.

18 August

14 August

12 August

6 August

5 August

  • The following Anthropic model is deprecated from DigitalOcean Inference as of 5 August 2026:

    Migrate to Claude Opus 4.8 (anthropic-claude-opus-4.8) to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

July 2026

27 July

24 July

23 July

  • The following OpenAI model is deprecated from DigitalOcean Inference as of 23 July 2026:

    Migrate to GPT-5.3-Codex (openai-gpt-5.3-codex) to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

  • We now support prompt caching for Gemma 4 for serverless inference.

  • The server-side model synthesis tool is available in public preview on DigitalOcean Inference. The model synthesis tool is an API-only multi-model tool that runs up to eight analysis models in parallel on the same task, compares their results, and returns one synthesized answer. You can use the model synthesis tool with serverless inference, dedicated inference, and inference routers on the Chat Completions, Responses, and Messages APIs. Opt in from the Feature Preview page. For more information, see How to Use the Model Synthesis Tool.

15 July

10 July

1 July

  • Claude Fable 5 is available again on DigitalOcean Inference for serverless inference, Agent Development Kit, and agents. For more information, see the Available Models page.

  • Prompt caching for open-source models in serverless inference chat completions and responses API is now in public preview. Open-source models cache context automatically, so you do not need to set the cache_control or prompt_cache_retention parameters.

    Prompt caching is available for the following open-source models:

    • DeepSeek V3.2
    • DeepSeek V4 Pro
    • DeepSeek V4 Flash
    • Kimi K2.5
    • Kimi K2.6
    • GLM 5
    • GLM-5.1
    • GLM-5.2
    • gpt-oss-120b
    • MiMo V2.5
    • MiMo V2.5 Pro
    • MiniMax M2.5
    • Qwen 3.5
    • Qwen3 Coder Flash

    For more information, see Use Prompt Caching.

June 2026

30 June

  • The following agent evaluation metrics are deprecated and should no longer be used:

    • Tone
    • Retrieved Chunk Usage
    • Prompt Perplexity

    Use the currently supported metrics listed in Agent Evaluation Metrics instead. To monitor deployed agent behavior outside of evaluations, use Agent Metrics and Runtime Logs.

  • The following Anthropic model is now available on DigitalOcean Inference for serverless inference, Agent Development Kit, and agents:

    For more information, see the Available Models page.

  • Agent evaluations support for the Agent Development Kit (ADK), previously in preview, is now removed.

    To evaluate agents, use agent evaluations via the DigitalOcean Control Panel for supported agent types. To monitor ADK agent behavior, use Agent Metrics and Runtime Logs.

  • Insights, agent tracing, and conversation logs are deprecated for all agents, including agents created through the Control Panel, CLI, API, and Agent Development Kit (ADK).

    To monitor deployed agent behavior, use Agent Metrics and Runtime Logs instead.

  • The Agent Evaluations MCP server tool has been renamed to Evaluations.

  • Custom metrics are now available for DigitalOcean Evaluations. You can define your own metrics to evaluate model behavior against criteria specific to your use case.

  • DigitalOcean Evaluations is now generally available. Use Evaluations to create test cases, run evaluation datasets, and measure model performance against selected metrics.

  • Presets are now available for DigitalOcean Evaluations. You can save and reuse evaluation configurations, including the candidate model, system prompt, hyperparameters, judge model, and metrics.

  • Model Evaluations is now renamed to DigitalOcean Evaluations.

29 June

  • Serverless Inference now requires a positive prepaid account balance before you can send inference requests. Usage charges are deducted from this balance, and access is suspended if it reaches $0. You can add a prepayment manually or enable auto-reload to replenish your balance automatically. For more information, see Manage Serverless Inference Prepayment.

24 June

22 June

17 June

  • The following Xiaomi model is now available on DigitalOcean Inference for serverless inference, dedicated inference, Agent Development Kit, and agents:

    For more information, see the Available Models page.

  • DigitalOcean Inference supports server-side tools on serverless inference, dedicated inference, and inference routers. You can add the following tools:

    • Web search, web fetch, knowledge base retrieval, and remote MCP server tools to your requests in the Chat Completions and Responses APIs.
    • Provider-native tools such as bash, text editor, computer use, and web fetch for Anthropic models with the Messages API.
    • Function calling and tool search for OpenAI models on the Responses API, and Anthropic models on the Messages API.

    Web search and web fetch are in public preview. For more information, see Use Server-Side Tools.

15 June

12 June

10 June

  • We support passthrough tool search on the Messages API for Anthropic models and the Responses API for OpenAI models, enabling deferred loading of tools in agentic workflows. There is no additional cost to using tool search. For more information, see Tool Search.

9 June

4 June

3 June

  • The public endpoint when creating a dedicated inference deployment is now disabled by default. For more information on how to enable access, see Use Dedicated Inference Endpoint.

May 2026

28 May

27 May

5 May

1 May

April 2026

28 April

  • The following Beijing Academy of Artificial Intelligence (BAAI) reranking model is now available on DigitalOcean Inference for DigitalOcean Knowledge Bases:

    For more information, see the Available Models page.

  • The following embeddings model is now available on DigitalOcean Inference for DigitalOcean Knowledge Bases:

    For more information, see the Available Models page.

  • Knowledge base enhancements are now generally available in DigitalOcean Inference, including the updated creation workflow, chunking controls, and data retrieval for testing knowledge base. For more information, see Create and Manage Agent Knowledge Bases.

  • RAG Playground is now available in DigitalOcean Inference for DigitalOcean Knowledge Bases. It lets you run queries against a knowledge base and test how a serverless inference model generates answers from retrieved content.

    For more information, see the DigitalOcean AI Platform Features page.

  • As part of the DigitalOcean AI-Native Cloud, DigitalOcean Gradient™ AI Platform is now DigitalOcean AI Platform.

  • DigitalOcean Inference now supports reranking for knowledge bases to improve the relevance of retrieved results before they’re returned or used in generated responses. For more information, see Create and Manage Agent Knowledge Bases and Test Reranking.

  • DigitalOcean Inference now lets you retrieve data from knowledge bases using the Control Panel with semantic, keyword, or hybrid searches, apply filters, review retrieved chunks, and copy live code examples. For more information, see Create and Manage Knowledge Bases.

  • Dedicated Inference is now in General Availability.

  • You can now browse Model Catalog through a DigitalOcean MCP server.

  • Batch inference lets you submit text-only batch jobs for OpenAI and Anthropic models. Using batch inference significantly reduces cost compared to real-time inference. For more information, see Use Batch Inference.

  • Bring Your Own Models (BYOM) is now available in Model Catalog. You can import models from Hugging Face or Spaces buckets or folders. For details, see Import a Model.

  • Model Catalog is now in General Availability.

  • You can now evaluate models available for serverless inference, inference routers, and dedicated inference deployments using a judge model. Scoring includes metrics such as correctness, completeness, ground truth faithfulness, and safety metrics. This feature is in public preview. You can opt in from the Feature Preview page. For more information, see Create a Dataset Evaluation.

  • We now support multimodal models for serverless inference. Multimodal models process and generate content across multiple data types, including images, audio, video, and text, thus enabling a much broader range of real-world applications, including document intelligence, voice agents, content generation, and accessibility tools. For more information, see Use Multimodal Inference.

  • The Model Playground now supports the following features when testing and comparing models:

    • Uploading images from local storage

    • Generating multimodal artifacts, such as images, audio, and text-to-speech, from models that support it

    Read Test and Compare Models for more information.

  • You can now use DigitalOcean personal access tokens for authenticating serverless inference requests. You can use a personal access token as an alternative to a model access key when sending requests to the serverless inference API. Model access keys remain recommended when you need per-application scoping, VPC restriction, or credentials dedicated to inference workloads. For more information, see Serverless Inference Overview.

  • The following models are now available on DigitalOcean Inference:

    For more information, see the Models page.

  • As part of the DigitalOcean AI-Native Cloud, DigitalOcean AI Inference Hub is now DigitalOcean Inference.

  • Inference Router in now available in public preview and enabled for all users. Using this feature, you can use multiple models in a model pool to configure routing rules and selection policy for inference requests. We provide pre-built templates or you can define custom task-matching logic using natural language, with configurable fallback support for reliability. For more information, see Inference Router.

  • DigitalOcean Inference now supports scoped model access keys. When you create a key, you can limit it to specific foundation models and inference routers, enable batch inference, and restrict it to a VPC network so that only requests from that VPC network can authenticate. Team owners can also view and manage keys created by other team members. Previously created keys continue to authenticate without changes. For more information, see Model Access Keys.

  • DigitalOcean Knowledge Base retrieval is now available through a DigitalOcean MCP server.

27 April

23 April

22 April

  • DigitalOcean Gradient™ AI Platform is now DigitalOcean AI Platform.

16 April

3 April

  • The following models are deprecated from DigitalOcean Gradient™ AI Platform:

    • Meta Llama 3.1 8B-Instruct
    • Mistral NeMo

    Migrate to a supported active model to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

  • The following models are deprecated from the Model Catalog:

    • Meta Llama 3.1 8B-Instruct
    • Mistral NeMo

    Migrate to Llama 3.3 70B-Instruct (llama3.3-70b-instruct) and gpt-oss-20b (openai-gpt-oss-20b) models respectively, to avoid service disruption.

2 April

1 April

March 2026

27 March

17 March

16 March

12 March

6 March

February 2026

27 February

  • We now support prompt caching for the following OpenAI models in serverless inference chat completions and responses API:

    • GPT-5.3-Codex
    • GPT-5.2
    • GPT-5.2 pro
    • GPT-5.1-Codex-Max
    • GPT-5
    • GPT-5 mini
    • GPT-5 nano
    • GPT-4.1
    • GPT-4o
    • GPT-4o mini
    • o1
    • o3
    • o3-mini
    • GPT-image-1

25 February

  • The following Anthropic model is deprecated from DigitalOcean Gradient™ AI Platform:

    Migrate to a supported active model to avoid service disruption. For information on our model deprecation policy and recommended replacement models, see Model Support Policy.

  • The following OpenAI models are now available on DigitalOcean Gradient™ AI Platform for serverless inference and Agent Development Kit:

    For more information, see the Available Models page.

24 February

  • You can now use reasoning with serverless inference chat completion. For more information, see Use Reasoning.

19 February

17 February

9 February

5 February

January 2026

30 January

December 2025

18 December

17 December

11 December

  • DigitalOcean Gradient™ AI Platform also now supports the Qwen3 Embedding 0.6B (Multilingual) model from Alibaba Qwen in public preview. Select this model for multilingual indexing workflows. Learn more on the embeddings models page.

9 December

October 2025

29 October

  • We support specifying a sitemap URL as a data source for your knowledge base. For more information, see Select Your Data Sources.

20 October

1 October

September 2025

25 September

  • You can now view activity logs for your knowledge bases in the DigitalOcean Control Panel. The Activity tab shows the 15 most recent indexing jobs, and includes details such as status, number of files or URLs processed, skipped, or failed, token usage, and charges. You can also download CSV files for more detailed debugging.

9 September

August 2025

18 August

12 August

11 August

5 August

July 2025

27 July

25 July

22 July

11 July

9 July

3 July

2 July

  • Agent tracing and conversation logs are now in public preview for DigitalOcean Gradient™ AI Platform. This allows you to review how your agents process prompts, including input and output content, tool calls, knowledge base retrievals, and processing times.

June 2025

5 June

  • Serverless inference is now available on DigitalOcean Gradient™ AI Platform. Serverless inference lets you to get direct responses from foundation models using a single API endpoint without creating an agent.

April 2025

29 April

28 April

23 April

22 April

  • You can now view the knowledge bases, functions, and guardrails that DigitalOcean Gradient™ AI Platform agents use to generate a response in the Agent Playground and the agent API.

  • You can now view the runtime logs for agents on DigitalOcean Gradient™ AI Platform. The logs display the events that occur during an agent’s execution, such as the knowledge bases and functions accessed to generate a response.

16 April

9 April

March 2025

25 March

February 2025

21 February

7 February

  • You can now use the DeepSeek-R1 model with agents on DigitalOcean Gradient™ AI Platform.

January 2025

31 January

  • You can now use Anthropic models with agents on DigitalOcean Gradient™ AI Platform.

28 January

24 January

  • The Sensitive Data Detection guardrail and its custom versions have been temporarily removed. To identify and anonymize sensitive data, add the following to your agent instructions:

    You must avoid providing responses containing sensitive or private information.

Sensitive information includes but is not limited to:

* Personal data (e.g., names, addresses, emails, phone numbers)
* Financial details (e.g., credit card numbers, bank accounts)
* Medical information
* Private communications
* Confidential business information

If the user's query involves sensitive information, respond with: "I'm sorry, I can't answer that."

22 January

We can't find any results for your search.

Try using different keywords or simplifying your search terms.