This feature is in public preview. To use it, opt in from the Feature Preview page.
How to Use Signals with Harness Runtimepublic
Last verified 12 Oct 2026
DigitalOcean Harness Runtime provides managed, hardware-isolated microVM sandboxes with built-in tools such as Chromium to run harnesses and execute arbitrary code. Rich lifecycle APIs preserve conversational history and working state across sessions, letting you pause, resume, and fork work to control costs and adapt to the nonlinear nature of agentic workflows. Scale complete agents such as Claude Code or use sandboxes independently for code execution, all through the same service. See What You Can Build for example use cases.
DigitalOcean Signals analyzes conversations with your Harness Runtime agents and detects behavioral signals, such as a user asking for a human, a failed tool call, an execution loop, or a user confirming that a response helped. Use Signals to find the conversations that went wrong without reading every transcript.
You manage Signals for Harness Runtime agents from the DigitalOcean Control Panel or with doctl version 1.181.0 or later. doctl needs a personal access token with the following scopes. A full-access token includes all of them.
signals:queryto read Signals data, create exports, and get or list deletion jobs.signals:updateto turn collection on or off.signals:deleteto request data deletion.
Signals applies consent and every other operation to the team that owns the token. For every command and flag, see the doctl signals reference.
Signals commands and API requests identify an environment by its agent ID, which is the ID of the Environment Config that the agent runs from. To list your configs and their IDs, run doctl harness-runtime config list. If you start a session with --spec and no saved config, Harness Runtime creates an Environment Config for that session automatically, and you use that config’s ID the same way.
How Signals Works
Signals organizes each conversation into the following levels:
- Dialogue: One completed run, made up of the user’s message and the steps taken to respond to it.
- Session: One conversation, made up of one or more dialogues.
- Signal: One behavioral pattern that Signals detects in a conversation, such as a user rephrasing a request or a tool call failing.
For each conversation, Signals writes a signal report. The report rates the conversation’s overall quality from Excellent to Severe, lists the signals it detected, and marks the conversation as concerning when it needs a closer look.
For what data Signals processes, how it redacts sensitive data, and where it stores data, see Signals data privacy.
Signals collects data only for environments that you enable it for. Collection starts with conversations that begin after you enable it, so Signals does not analyze earlier conversations.
Signal Types
Signals detects 25 signal types in seven categories, grouped into three layers: interaction, execution, and environment. In API responses, a signal’s layer field contains the layer name in lowercase, and its category field contains the value in the Category column.
Interaction Signals
Interaction signals describe how the user responds to the conversation.
| Category | Signal type | Description |
|---|---|---|
misalignment |
MisalignmentClarification |
The user asks a clarifying question about a response. |
misalignment |
MisalignmentCorrection |
The user corrects the understanding of the task. |
misalignment |
MisalignmentRephrase |
The user restates their request in different words. |
stagnation |
StagnationDragging |
The conversation continues without meaningful progress toward the user’s goal. |
stagnation |
StagnationRepetition |
A response repeats information that was already provided. |
disengagement |
DisengagementEscalation |
The user asks to speak with a human or to escalate. |
disengagement |
DisengagementNegativeStance |
The user expresses frustration, criticism, or dissatisfaction. |
disengagement |
DisengagementQuit |
The user attempts to end or abandon the conversation. |
satisfaction |
SatisfactionConfirmation |
The user confirms that a response is correct or helpful. |
satisfaction |
SatisfactionGratitude |
The user expresses thanks or appreciation. |
satisfaction |
SatisfactionSuccess |
The user’s goal was achieved. |
Execution Signals
Execution signals show whether tool calls and actions succeed.
| Category | Signal type | Description |
|---|---|---|
failure |
ExecutionFailureAuthMisuse |
An action was attempted with incorrect or insufficient credentials. |
failure |
ExecutionFailureBadQuery |
An invalid query or request was constructed. |
failure |
ExecutionFailureInvalidArgs |
Invalid arguments were passed to a tool or function. |
failure |
ExecutionFailureStateError |
An action was attempted that is invalid in the current state. |
failure |
ExecutionFailureToolNotFound |
A tool was called that does not exist or is unavailable. |
loops |
ExecutionLoopsOscillation |
Two or more actions alternate without making progress. |
loops |
ExecutionLoopsParameterDrift |
The same action is retried with slightly different parameters each time. |
loops |
ExecutionLoopsRetry |
The same action is retried repeatedly without success. |
Environment Signals
Environment signals show whether the infrastructure and external services around the agent hold up.
| Category | Signal type | Description |
|---|---|---|
exhaustion |
EnvironmentExhaustionApiError |
An external API returned an error. |
exhaustion |
EnvironmentExhaustionContextOverflow |
The conversation exceeded the model’s context window. |
exhaustion |
EnvironmentExhaustionMalformed |
A response was malformed or could not be parsed. |
exhaustion |
EnvironmentExhaustionNetwork |
A network connection failed. |
exhaustion |
EnvironmentExhaustionRateLimit |
An external service rate-limited the request. |
exhaustion |
EnvironmentExhaustionTimeout |
An operation timed out before it completed. |
Enable Signals for an Environment
Enabling Signals for an environment records your consent for Signals to collect and analyze that environment’s conversations. The change can take about a minute to take effect.
Enable Signals Using the Control Panel
Go to the Signals tab of the Harness Runtime page. The tab appears after you create your first environment. The Signals tab calls environments agents.
Select the environment from the drop-down menu, and then click the Signals Disabled toggle.
Enable Signals Using doctl
Run doctl signals consent set with the environment’s agent ID. Write the flag as --enabled=true, with an equals sign:
doctl signals consent set --agent-id <agent-id> --enabled=trueThe output looks similar to the following:
Output
ID Team ID Source Agent ID Enabled Updated At
42 1234567 agent 3f2a1c4e-5b6d-4e7f-8a9b-0c1d2e3f4a5b true 2026-10-01T12:00:00ZCheck Which Environments Have Signals Enabled
To list the Signals status of every environment on your team, run:
doctl signals consent listThe output looks similar to the following:
Output
ID Team ID Source Agent ID Enabled Updated At
42 1234567 agent 3f2a1c4e-5b6d-4e7f-8a9b-0c1d2e3f4a5b true 2026-10-01T12:00:00Z
43 1234567 agent a1b2c3d4-e5f6-7890-abcd-ef1234567890 false 2026-10-02T09:30:00ZTo check one environment, run:
doctl signals consent get --agent-id <agent-id>The output looks similar to the following:
Output
ID Team ID Source Agent ID Enabled Updated At
42 1234567 agent 3f2a1c4e-5b6d-4e7f-8a9b-0c1d2e3f4a5b true 2026-10-01T12:00:00ZThe Enabled column shows whether collection is on for the environment. An environment that you have never enabled Signals for shows false.
Disable Signals for an Environment
Disabling Signals stops collection for new conversations. It does not delete data that Signals has already collected. To delete that data, see Signals Data Deletion Request.
Disable Signals Using the Control Panel
Go to the Signals tab of the Harness Runtime page, select the environment from the drop-down menu, and then click the Signals Enabled toggle. In the Disable Signals window, enter the environment’s name in the Agent name field, and then click Disable.
Disable Signals Using doctl
Set --enabled=false. Include the equals sign. If you write --enabled false without it, doctl reads the flag as true and enables collection instead:
doctl signals consent set --agent-id <agent-id> --enabled=falseView Sessions for an Environment
To list an environment’s sessions, run doctl signals session list with the agent ID:
doctl signals session list --agent-id <agent-id>Sessions are sorted by most recent activity first. The output includes the following columns:
| Column | Description |
|---|---|
Session ID |
The session’s unique identifier. Use it to view the session’s dialogues. |
Total Turns |
The number of turns in the session. |
Started At |
When the session started, in ISO 8601 format. |
Duration (s) |
The time from the start of the session to its most recent activity, in seconds. |
Signal Count |
The number of signals in the session’s most recent signal report. |
To narrow the list, add any of the following flags:
--signal-type: Return only sessions that contain the given signal type. Repeat the flag to match any of several types.--start-timeand--end-time: Limit the results to a time window, in Unix epoch seconds.--start-timeis inclusive and--end-timeis exclusive. The filter matches sessions by their last activity, not their start time.--limit: The number of sessions to return, from 1 to 100. The default is 20.
doctl returns one page of results. To see sessions beyond the first page, narrow the time window with --start-time and --end-time.
View the Dialogues in a Session
To list the dialogues in a session, run doctl signals session dialogues with a session ID from the session list:
doctl signals session dialogues --session-id <session-id>Dialogues are listed in the order they occurred. The output shows each dialogue’s ID, its sequence in the session, its run status, the first 80 characters of the user’s message, and the types of any signals detected in that run.
To see the full user message, the steps taken to respond, and the details of each signal, add --output json:
doctl signals session dialogues --session-id <session-id> --output jsonThe JSON output contains one object for each dialogue:
[
{
"id": 1205818346715843946,
"run_id": "5b4a3c2d-1e0f-4a9b-8c7d-6e5f4a3b2c1d",
"created_at": "2026-09-30T19:42:03Z",
"sequence": 2,
"user_message": "Deploy the latest build to staging.",
"steps": [
{"seq": 1, "type": "agent_reply", "text": "I couldn't find a deployment tool."}
],
"run_status": "completed",
"failure_code": null,
"signals": [
{
"signal_type": "ExecutionFailureToolNotFound",
"label": "Tool not found",
"run_uuid": "5b4a3c2d-1e0f-4a9b-8c7d-6e5f4a3b2c1d",
"step_index": 1,
"confidence": 0.92,
"snippet": null,
"layer": "execution",
"category": "failure"
}
]
}
]The main fields in each dialogue are:
| Field | Description |
|---|---|
id |
The dialogue’s identifier. Some JSON parsers can’t represent large 64-bit integers exactly, so use a parser that preserves them. |
run_id |
The identifier of the run that produced the dialogue. |
sequence |
The dialogue’s position within the session. |
user_message |
The message the user sent. |
steps |
The steps taken to respond, in order. |
run_status |
completed, incomplete, or failed. |
signals |
The signals detected in this run. Each entry includes the signal type, its layer and category, and a confidence value from 0 to 1. The field is omitted when the run has no signals. |
You can filter dialogues with --signal-type, --start-time, --end-time, and --limit, the same way as sessions.
Export Signal Data
Exports package an environment’s sessions, signals, and dialogues into a compressed JSON file that you can download for offline analysis. When you export Signals data, Signals also creates Simulation scenarios from those conversations. Exports run in the background, so you create an export, wait for it to complete, and then request a download URL.
Exports remove personal information from free text, such as user messages, step text, and signal snippets.
Create an Export
To create an export of all of an environment’s sessions, run:
doctl signals export create --agent-id <agent-id>To narrow the export, add --signal-type or --session-ids, both repeatable, or a time window with --start-time and --end-time in Unix epoch seconds. --start-time is inclusive and --end-time is exclusive. To list the signal types you can filter on, run doctl signals export options.
The signal type filter chooses which signals appear in the export. It does not remove sessions, so a session with no matching signals still appears with an empty signals list and its full dialogues. The time window applies to session activity.
If an identical export is already queued or running, the command returns that export instead of creating a new one.
The output shows the new export’s ID, status, signal type filter, and creation time. Export timestamps are Unix epoch seconds.
Check an Export’s Status
To check an export, run:
doctl signals export get <export-id>The Status column has one of the following values:
| Status | Description |
|---|---|
queued |
The export is waiting to be processed. |
running |
The export file is being built. |
complete |
The export file is ready to download. |
failed |
The export failed. To see the reason, run the command again with --output json and check the error_message field. |
expired |
The export file passed its 7-day retention period and can no longer be downloaded. |
Small exports typically complete within seconds.
Download an Export
When the status is complete, request a download URL:
doctl signals export download <export-id>The output shows a pre-signed download URL and the time it expires, in Unix epoch seconds. The URL is valid for 15 minutes. To get a new one, run the command again. The command returns a 404 error if the export is not complete yet, and a 410 error after the export expires.
To download and decompress the file, run the following commands, replacing <download-url> with the URL from the output:
curl -o signals-export.json.gz "<download-url>"
gunzip signals-export.json.gzThe file contains a nested JSON structure of sessions and their signals and dialogues, ordered from oldest to newest.
List Exports
To list your team’s exports, run doctl signals export list. Add --agent-id to list one environment’s exports. The command returns 20 exports by default. Use --limit to return from 1 to 100.
Signals Data Deletion Request
Deleting Signals data permanently and irreversibly removes the conversation data and signal reports that Signals collected for the requested scope. Deletion cannot be undone. When you create a deletion, Signals immediately turns collection off for that scope, hides matching conversations from Signals results, then deletes that conversation data asynchronously in the background. You create a job, poll until it finishes, then check the result.
For Harness Runtime (--type agent), deletion applies to one agent. Set --agent-id to the agent’s Environment Config ID (the agent UUID). On create, Signals hides that agent’s conversations from Signals results and turns collection off for the agent. To resume collection, enable Signals for the environment again. You can do this before the deletion finishes.
Deletion does not remove export files that you created earlier. They remain available to download until they expire (7 days).
Aggregated Insights or metrics charts may still show historical summary data points after deletion. Those summaries are not the same as recoverable conversation content.
Create a Deletion Request
To permanently delete the data, run doctl signals deletion create with the agent type and the environment’s Environment Config ID (--agent-id):
doctl signals deletion create --type agent --agent-id <environment-config-id>doctl asks you to confirm before it sends the request. Add --force to skip the prompt, for example in a script.
The output shows the deletion ID, type, status, and creation time. Timestamps are Unix epoch seconds.
If a deletion for the same target is already queued or running, the command returns that job instead of creating a new one. Each team can have up to two deletions queued or running at a time. If you reach that limit, the command returns a 429 error. Wait for a job to finish, then try again.
Get Deletion Request Status
To check a deletion, run:
doctl signals deletion get <deletion-id>The Status column has one of the following values:
| Status | Description |
|---|---|
queued |
The deletion is waiting to start. |
running |
Signals is deleting the data. |
complete |
The data is permanently deleted. |
failed |
The deletion failed. Matching conversations stay hidden and collection stays off. Run the command again with --output json and check error_message, then create the deletion again to retry. |
Deletion runs asynchronously in the background. The time to complete depends on how much data needs to be removed. DigitalOcean may retry transient processing errors while a job is still active. If a job ends as failed, create the deletion again to retry.
List Deletion Requests
To list your team’s deletions, newest first, run:
doctl signals deletion listThe command returns 20 deletions by default. Use --limit to return from 1 to 100. If there are more results, doctl prints a next-page token. Pass it with --after <token> to fetch the next page.
Limits
-
Signals is in public preview, so its limits and behavior may change.
-
Accounts in a free trial can’t use Signals.
-
Each team can have up to two deletions queued or running at a time.
-
Completed exports are available to download for 7 days.
-
List commands return up to 100 items.
-
The Signals API uses the standard DigitalOcean API rate limits.