DigitalOcean Evaluations

Generated on 12 Oct 2026

DigitalOcean Evaluations measure model, Inference Router, and agent performance using dataset evaluations and simulations.

Getting Started

Quickstarts and intermediate tutorials to get started.

How-Tos

How to accomplish specific tasks in detail, like creation/deletion, configuration, and management.

Reference

Native and third-party tools, troubleshooting, and answers to frequently asked questions.

Concepts

Explanations and definitions of core concepts in Evaluations.

Details

Features, plans and pricing, availability, limits, known issues, and more.

Latest Updates

12 October 2026

  • Simulation is now available in public preview. Simulation tests agent behavior through generated, multi-turn conversations using scenario sets, user simulator models, judge models, and conversational metrics.

30 June 2026

  • Model Evaluations is now renamed to DigitalOcean Evaluations.

  • Presets are now available for DigitalOcean Evaluations. You can save and reuse evaluation configurations, including the candidate model, system prompt, hyperparameters, judge model, and metrics.

  • DigitalOcean Evaluations is now generally available. Use Evaluations to create test cases, run evaluation datasets, and measure model performance against selected metrics.

  • Custom metrics are now available for DigitalOcean Evaluations. You can define your own metrics to evaluate model behavior against criteria specific to your use case.

  • The Agent Evaluations MCP server tool has been renamed to Evaluations.

  • Agent evaluations support for the Agent Development Kit (ADK), previously in preview, is now removed.

    To evaluate agents, use agent evaluations via the DigitalOcean Control Panel for supported agent types. To monitor ADK agent behavior, use Agent Metrics and Runtime Logs.

  • The following agent evaluation metrics are deprecated and should no longer be used:

    • Tone
    • Retrieved Chunk Usage
    • Prompt Perplexity

    Use the currently supported metrics listed in Agent Evaluation Metrics instead. To monitor deployed agent behavior outside of evaluations, use Agent Metrics and Runtime Logs.

28 April 2026

  • You can now evaluate models available for serverless inference, inference routers, and dedicated inference deployments using a judge model. Scoring includes metrics such as correctness, completeness, ground truth faithfulness, and safety metrics. This feature is in public preview. You can opt in from the Feature Preview page. For more information, see Create a Dataset Evaluation.

For more information, see all Evaluations release notes.

We can't find any results for your search.

Try using different keywords or simplifying your search terms.