For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Usage

client.genai.create_simulation_run(
    body={
        "agent_config": {...},
        "evaluation_config": {...},
        "exploration_budget": 123,
        ...,
    },
)

Description

To create a simulation run, send a POST request to /v2/gen-ai/simulation_runs. The scenario set must be ready. Judge and user simulator models are optional and use platform defaults when omitted. The run is returned in the pending state.

Parameters

agent_config object optional

Configuration of the candidate agent under test for a simulation run.

Show child properties

agent_deployment_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

Optional agent deployment to run. Defaults to the agent's current deployment when unset.

agent_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

UUID of the agent to exercise as the candidate.

name string optional

Example: example name

Display name of the candidate agent. Persisted with the run.

evaluation_config object optional

Optional configuration that opts a simulation run into an evaluation. When
included on the create-run request, the platform reserves a
model_evaluation_run linked to this simulation run and dispatches the
evaluation automatically after the simulation finishes. The evaluation
reuses the simulation run's judge_model_uuid.

Show child properties

metric_uuids array of strings optional

Example: ['example string']

Evaluation metric UUIDs (from the evaluation metrics catalog). Must be
simulation-eligible (multi-turn) metrics; the create request is rejected
if any UUID is unknown or ineligible.

star_metric object optional

Show child properties

metric_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

name string optional

Example: example name

success_threshold number optional

Example: 123

The success threshold for the star metric.
This is a value that the metric must reach to be considered successful.

success_threshold_pct integer optional

Example: 123

The success threshold for the star metric.
This is a percentage value between 0 and 100.

exploration_budget integer optional

Example: 123

Optional run-level journeys-per-scenario override. When set, overrides each scenario's exploration_budget.

judge_model_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

Optional model override for the judge. Platform default when unset.

max_turns integer optional

Example: 123

Optional run-level turn budget. When set, overrides each scenario's max_turns.

name string optional

Example: example name

Optional run name.

scenario_set_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

UUID of the existing scenario set to execute. The set must be ready.

user_simulator_config object optional

Optional user simulator model settings such as temperature and max_tokens.

user_simulator_model_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

Optional model override for the user simulator. Platform default when unset.

Request Sample

import os
from pydo import Client

client = Client(token=os.environ.get("DIGITALOCEAN_TOKEN"))

req = {
  "agent_config": {
    "agent_deployment_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "agent_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "name": "example name"
  },
  "evaluation_config": {
    "metric_uuids": [
      "example string"
    ],
    "star_metric": {
      "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "name": "example name",
      "success_threshold": 123,
      "success_threshold_pct": 123
    }
  },
  "exploration_budget": 123,
  "judge_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "max_turns": 123,
  "name": "example name",
  "scenario_set_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "user_simulator_model_uuid": "123e4567-e89b-12d3-a456-426614174000"
}

resp = client.genai.create_simulation_run(body=req)

Response Example

{
  "simulation_run": {
    "agent_config": {
      "agent_deployment_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "agent_uuid": "123e4567-e89b-12d3-a456-426614174000",
      "name": "example name"
    },
    "created_at": "2023-01-01T00:00:00Z",
    "created_by_user_email": "[email protected]",
    "created_by_user_id": "12345",
    "deleted_at": "2023-01-01T00:00:00Z",
    "evaluation_run_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "exploration_budget": 123,
    "failure_reason": "example string",
    "journeys_finished": 123,
    "judge_model_name": "example name",
    "judge_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "max_turns": 123,
    "name": "example name",
    "result_summary": {
      "total_duration_sec": "12345"
    },
    "run_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "scenario_count": 123,
    "scenario_set_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "status": "SIMULATION_RUN_STATUS_UNSPECIFIED",
    "total_journeys": 123,
    "updated_at": "2023-01-01T00:00:00Z",
    "user_simulator_model_name": "example name",
    "user_simulator_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "workflow_uuid": "123e4567-e89b-12d3-a456-426614174000"
  }
}

More Information

See /v2/gen-ai/simulation_runs in the API reference for additional detail on responses, headers, parameters, and more.