---
title: DigitalOcean Evaluations Features
description: Features for evaluating model, Inference Router, and agent performance using dataset evaluations and simulations.
product: Evaluations
url: https://docs.digitalocean.com/products/evaluations/details/features/
last_updated: "2026-10-13"
---

> **For AI agents:** The documentation index is at [https://docs.digitalocean.com/llms.txt](https://docs.digitalocean.com/llms.txt). Markdown versions of pages use the same URL with `index.html.md` in place of the HTML page (for example, append `index.html.md` to the directory path instead of opening the HTML document).

# DigitalOcean Evaluations Features

DigitalOcean Evaluations measure model, Inference Router, and agent performance using dataset evaluations and simulations.

Evaluations help you measure and compare model, Inference Router, and agent performance using repeatable evaluation workflows.

## Dataset Evaluations

Dataset evaluations test a model, Inference Router, or supported agent against a fixed dataset of test inputs.

You can use dataset evaluations for:

- **Single-turn testing**: Send independent prompts to a candidate and score its generated responses.
- **Multi-turn testing**: Score existing conversations contained in a dataset without selecting a candidate model.
- **Ground truth evaluation**: Compare candidate responses against expected answers included in the dataset.
- **Custom evaluation criteria**: Define custom metrics when built-in metrics do not cover your use case.
- **Evaluation comparison**: Compare two dataset evaluations across overall scores, star metrics, pass thresholds, configurations, metric pass rates, token usage, latency, Inference Router task results when applicable, and per-entry results when both evaluations use the same dataset.

Dataset evaluation results include overall and per-metric scores, judge rationale, candidate and judge token usage, and candidate latency. You can also download evaluation results, rename evaluation runs, and delete evaluations from the **Evaluations & Simulations** page.

## Simulation (public)

Simulations test agent behavior through generated, multi-turn conversations.

You can evaluate:

- An existing managed agent with its configured instructions, tools, and settings.
- An agent that the simulation creates from a Serverless Inference model, third-party model, Dedicated Inference deployment, or Inference Router.

Simulations use scenario sets that define situations for a user simulator and agent to act out. You can generate scenario sets from a goal description, upload them as JSONL files, import ready-made sets from the Common Scenario & Goal Library, or import real agent conversations where DigitalOcean Signals detected a problem. You can configure the user simulator model, judge model, evaluation metrics (including custom metrics), turn limit, and number of episodes for each scenario.

You can create and manage scenario sets and simulation runs in the Control Panel or using the [DigitalOcean API](https://docs.digitalocean.com/reference/api/reference/agent-platform-api/index.html.md).

Simulation results include scenario and episode status, duration, token usage, conversation trajectories, and evaluation metric results.

## Evaluation Metrics

Built-in metrics let you evaluate response quality, safety, conversation behavior, tool use, and goal completion.

Single-turn evaluation metrics include **Correctness**, **Completeness**, **Ground Truth Faithfulness**, **Answer Relevancy**, **Coherence**, **Clarity**, and **Harmfulness**.

Multi-turn and simulation metrics include **User goal completion**, **Knowledge Retention**, **Conversation Completeness**, **Turn Relevancy**, **Tool Use Accuracy**, and **Agent Policy Adherence**.

For dataset evaluations and simulations, you can also create custom metrics with your own scoring prompts. Custom metrics for single-turn dataset evaluations can also require ground truth values from the dataset.

## Reusable Evaluation Configurations

Evaluation presets let you save and reuse evaluation configurations across runs.

Dataset evaluation presets can retain settings such as the dataset, judge model, evaluation metrics, and star metric. Simulation presets can retain the scenario set, agent, user simulator model, judge model, evaluation metrics, turn limit, and episodes per scenario. Each preset applies only to its own evaluation type. Reusing configurations helps keep evaluation settings consistent when comparing changes between runs.