For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Usage

client.genai.create_model_evaluation_run(
    body={
        "candidate_inference_config": {...},
        "candidate_model_name": "example name",
        "candidate_model_source": "CANDIDATE_MODEL_SOURCE_SERVERLESS",
        ...,
    },
)

Description

To create a model evaluation run, send a POST request to /v2/genai/model_evaluation_runs.

Parameters

candidate_inference_config object optional

Inference configuration for the candidate model during evaluation.

Show child properties

max_tokens integer optional

Example: 123

stop_token string optional

Example: example string

system_prompt string optional

Example: example string

temperature number optional

Example: 123

candidate_model_name string optional

Example: example name

Model slug used to call the candidate model API.
For dedicated inference, this is the model slug from the deployment.
For serverless, this should match the model's internal name.

candidate_model_source string optional

Whether inference runs against the serverless platform, a dedicated deployment, or a model router.

One of: CANDIDATE_MODEL_SOURCE_SERVERLESS, CANDIDATE_MODEL_SOURCE_DEDICATED, CANDIDATE_MODEL_SOURCE_ROUTER

Default: CANDIDATE_MODEL_SOURCE_SERVERLESS

candidate_model_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

UUID of the candidate model to evaluate.

dataset_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

UUID of the dataset to use for evaluation.

eval_preset_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

judge_model_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

UUID of the judge model used to score responses.

metric_uuids array of strings optional

Example: ['example string']

UUIDs of metrics to evaluate (selected from ListModelEvaluationMetrics).

name string optional

Example: example name

preset_name string optional

Example: example name

preset_save_sections array of strings optional

Example: ['PRESET_SAVE_SECTION_CANDIDATE', 'PRESET_SAVE_SECTION_METRICS']

Which sections of this run's resolved configuration to persist as a
reusable preset. Each selected section saves only its own fields; the
remaining sections stay empty on the preset and must be supplied inline
on future runs that reference it. Empty means do not save a preset
(unless the deprecated save_as_preset boolean is true, in which case
all sections are saved). Ignored when eval_preset_uuid is set. Use
preset_name to label the saved preset.

save_as_preset boolean optional

Example: True

Deprecated: use preset_save_sections. When true and
preset_save_sections is empty, all five sections of the resolved
configuration are saved as a reusable preset (legacy behavior). Ignored
when eval_preset_uuid is set.

source string optional

Example: example string

Source of the run creation (api, sdk, cli).

star_metric object optional

Show child properties

metric_uuid string optional

Example: 123e4567-e89b-12d3-a456-426614174000

name string optional

Example: example name

success_threshold number optional

Example: 123

The success threshold for the star metric.
This is a value that the metric must reach to be considered successful.

success_threshold_pct integer optional

Example: 123

The success threshold for the star metric.
This is a percentage value between 0 and 100.

Request Sample

import os
from pydo import Client

client = Client(token=os.environ.get("DIGITALOCEAN_TOKEN"))

req = {
  "candidate_inference_config": {
    "max_tokens": 123,
    "stop_token": "example string",
    "system_prompt": "example string",
    "temperature": 123
  },
  "candidate_model_name": "example name",
  "candidate_model_source": "CANDIDATE_MODEL_SOURCE_SERVERLESS",
  "candidate_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "dataset_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "eval_preset_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "judge_model_uuid": "123e4567-e89b-12d3-a456-426614174000",
  "metric_uuids": [
    "example string"
  ],
  "name": "example name",
  "preset_name": "example name",
  "preset_save_sections": [
    "PRESET_SAVE_SECTION_CANDIDATE",
    "PRESET_SAVE_SECTION_METRICS"
  ],
  "save_as_preset": True,
  "source": "example string",
  "star_metric": {
    "metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
    "name": "example name",
    "success_threshold": 123,
    "success_threshold_pct": 123
  }
}

resp = client.genai.create_model_evaluation_run(body=req)

Response Example

{
  "eval_run_uuid": "123e4567-e89b-12d3-a456-426614174000"
}

More Information

See /v2/gen-ai/model_evaluation_runs in the API reference for additional detail on responses, headers, parameters, and more.