For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).
Usage
client.v2.genAi.custom_evaluation_metrics.post(body: ApiCreateCustomEvaluationMetricInputPublic, requestConfiguration?: RequestConfiguration<object>): Promise<undefined | ApiCreateCustomEvaluationMetricOutput>
Description
To create a custom LLM-as-judge metric for model evaluation, send a POST request to /v2/gen-ai/custom_evaluation_metrics.
Parameters
config object optionalConfiguration for a custom model-evaluation metric scored by an LLM judge.
Prompt and model response are always included in the judge context.
Show child properties
createdAt string optionalExample: 2023-01-01T00:00:00Z
deletedAt string optionalExample: 2023-01-01T00:00:00Z
When set, the custom metric is soft-deleted and must not appear in pickers.
requiresGroundTruth boolean optionalExample: true
When true, each row must provide ground truth and it is included in the judge context.
When false, ground truth is not required and is not sent to the judge.
scoringPrompt string optionalExample: example string
Instructions for the judge model (multi-line).
updatedAt string optionalExample: 2023-01-01T00:00:00Z
description string optionalExample: "Scores adherence to our support macros"
metricName string optionalExample: "My domain tone metric"
Request Sample
Show Request Sample
import { FetchRequestAdapter } from "@microsoft/kiota-http-fetchlibrary";
import {
createDigitalOceanClient,
DigitalOceanApiKeyAuthenticationProvider,
} from "@digitalocean/dots";
const authProvider = new DigitalOceanApiKeyAuthenticationProvider(process.env.DIGITALOCEAN_TOKEN!);
const adapter = new FetchRequestAdapter(authProvider);
const client = createDigitalOceanClient(adapter);
const response = await client.v2.genAi.custom_evaluation_metrics.post({
config: {
createdAt: new Date("2023-01-01T00:00:00Z"),
deletedAt: new Date("2023-01-01T00:00:00Z"),
requiresGroundTruth: true,
scoringPrompt: "example string",
updatedAt: new Date("2023-01-01T00:00:00Z"),
},
description: "\"Scores adherence to our support macros\"",
metricName: "\"My domain tone metric\"",
});
Response Example
Show Response Example
{
"metric": {
"associated_presets": [],
"category": "METRIC_CATEGORY_UNSPECIFIED",
"custom_eval_config": {
"created_at": "2023-01-01T00:00:00Z",
"deleted_at": "2023-01-01T00:00:00Z",
"requires_ground_truth": true,
"scoring_prompt": "example string",
"updated_at": "2023-01-01T00:00:00Z"
},
"description": "example string",
"evaluation_scope": "EVALUATION_SCOPE_UNSPECIFIED",
"inverted": true,
"is_metric_goal": true,
"metric_name": "example name",
"metric_rank": 123,
"metric_type": "METRIC_TYPE_UNSPECIFIED",
"metric_uuid": "123e4567-e89b-12d3-a456-426614174000",
"metric_value_type": "METRIC_VALUE_TYPE_UNSPECIFIED",
"range_max": 123,
"range_min": 123,
"source": "EVALUATION_METRIC_SOURCE_UNSPECIFIED"
}
}
See /v2/gen-ai/custom_evaluation_metrics in the API reference for additional detail on responses, headers, parameters, and more.