How to Use Signals with Serverless Inferencepublic

Last verified 12 Oct 2026

Inference provides a single control plane for managing inference workflows. It includes a Model Catalog where you can view available foundation models, including both DigitalOcean-hosted and third-party commercial models, compare model capabilities and pricing, use routing to match inference requests to the best-fit model, and run inference using serverless or dedicated deployments.

DigitalOcean Signals analyzes chat conversations that your applications send to Serverless Inference and detects behavioral signals, such as a user asking for a human, a failed tool call, an execution loop, or a user confirming that a response helped. Use Signals to find the conversations that went wrong without reading every transcript.

You manage Signals with doctl version 1.181.0 or later. doctl needs a personal access token with the following scopes. A full-access token includes all of them.

  • signals:query to read Signals data, create exports, and get or list deletion jobs.
  • signals:update to turn collection on or off.
  • signals:delete to request data deletion.

Signals applies consent and every other operation to the team that owns the token. For every command and flag, see the doctl signals reference.

How Signals Works

Signals organizes each conversation into the following levels:

  • Dialogue: One completed run, made up of the user’s message and the steps taken to respond to it.
  • Session: One conversation, made up of one or more dialogues.
  • Signal: One behavioral pattern that Signals detects in a conversation, such as a user rephrasing a request or a tool call failing.

For each conversation, Signals writes a signal report. The report rates the conversation’s overall quality from Excellent to Severe, lists the signals it detected, and marks the conversation as concerning when it needs a closer look.

For what data Signals processes, how it redacts sensitive data, and where it stores data, see Signals data privacy.

Signal Types

Signals detects 25 signal types in seven categories, grouped into three layers: interaction, execution, and environment. In API responses, a signal’s layer field contains the layer name in lowercase, and its category field contains the value in the Category column.

Interaction Signals

Interaction signals describe how the user responds to the conversation.

Category Signal type Description
misalignment MisalignmentClarification The user asks a clarifying question about a response.
misalignment MisalignmentCorrection The user corrects the understanding of the task.
misalignment MisalignmentRephrase The user restates their request in different words.
stagnation StagnationDragging The conversation continues without meaningful progress toward the user’s goal.
stagnation StagnationRepetition A response repeats information that was already provided.
disengagement DisengagementEscalation The user asks to speak with a human or to escalate.
disengagement DisengagementNegativeStance The user expresses frustration, criticism, or dissatisfaction.
disengagement DisengagementQuit The user attempts to end or abandon the conversation.
satisfaction SatisfactionConfirmation The user confirms that a response is correct or helpful.
satisfaction SatisfactionGratitude The user expresses thanks or appreciation.
satisfaction SatisfactionSuccess The user’s goal was achieved.

Execution Signals

Execution signals show whether tool calls and actions succeed.

Category Signal type Description
failure ExecutionFailureAuthMisuse An action was attempted with incorrect or insufficient credentials.
failure ExecutionFailureBadQuery An invalid query or request was constructed.
failure ExecutionFailureInvalidArgs Invalid arguments were passed to a tool or function.
failure ExecutionFailureStateError An action was attempted that is invalid in the current state.
failure ExecutionFailureToolNotFound A tool was called that does not exist or is unavailable.
loops ExecutionLoopsOscillation Two or more actions alternate without making progress.
loops ExecutionLoopsParameterDrift The same action is retried with slightly different parameters each time.
loops ExecutionLoopsRetry The same action is retried repeatedly without success.

Environment Signals

Environment signals show whether the infrastructure and external services around the agent hold up.

Category Signal type Description
exhaustion EnvironmentExhaustionApiError An external API returned an error.
exhaustion EnvironmentExhaustionContextOverflow The conversation exceeded the model’s context window.
exhaustion EnvironmentExhaustionMalformed A response was malformed or could not be parsed.
exhaustion EnvironmentExhaustionNetwork A network connection failed.
exhaustion EnvironmentExhaustionRateLimit An external service rate-limited the request.
exhaustion EnvironmentExhaustionTimeout An operation timed out before it completed.

Consent for Serverless Inference works differently from consent for Harness Runtime environments:

  • One consent setting covers every Serverless Inference request made with your team’s keys. You can’t enable Signals for an individual model, key, or endpoint.
  • Collection is off until you enable it. Without consent, Serverless Inference processes your requests normally, but Signals does not store them.
  • Enabling consent starts collection for new requests. Signals does not analyze earlier traffic. Disabling consent stops collection for new requests within about a minute, but does not delete data that Signals already collected. To remove that data, see Signals Data Deletion Request.
  • Serverless Inference consent is separate from Harness Runtime environment consent. Enabling one does not enable the other.

Enable Signals for Serverless Inference

To enable Signals for your team’s Serverless Inference requests, run doctl signals consent set with --source inference. Write the flag as --enabled=true, with an equals sign:

doctl signals consent set --source inference --enabled=true

The output shows the consent record with Source set to inference and Enabled set to true.

Check Whether Signals Is Enabled for Serverless Inference

To check your team’s Serverless Inference consent, run:

doctl signals consent get --source inference

The Enabled column shows whether collection is on. If your team has never enabled Signals for Serverless Inference, it shows false.

To see every consent record for your team, including Serverless Inference and any Harness Runtime environments, run:

doctl signals consent list

To list only the Serverless Inference record, add --source inference:

doctl signals consent list --source inference

Disable Signals for Serverless Inference

To disable Signals for your team’s Serverless Inference requests, run:

doctl signals consent set --source inference --enabled=false

Signals keeps the consent record with Enabled set to false, so you have a record of when consent was granted and withdrawn. Collection stops for new requests within about a minute.

View Serverless Inference Sessions

To list your team’s Serverless Inference sessions, run doctl signals session list with --type inference:

doctl signals session list --type inference

Sessions are sorted by most recent activity first. The output includes the following columns:

Column Description
Session ID The session’s unique identifier. Use it to view the session’s dialogues.
Total Turns The number of turns in the session.
Started At When the session started, in ISO 8601 format.
Duration (s) The time from the start of the session to its most recent activity, in seconds.
Signal Count The number of signals in the session’s most recent signal report.

To narrow the list, add any of the following flags:

  • --signal-type: Return only sessions that contain the given signal type. Repeat the flag to match any of several types.
  • --start-time and --end-time: Limit the results to a time window, in Unix epoch seconds. --start-time is inclusive and --end-time is exclusive. The filter matches sessions by their last activity, not their start time.
  • --limit: The number of sessions to return, from 1 to 100. The default is 20.

doctl returns one page of results. To see sessions beyond the first page, narrow the time window with --start-time and --end-time.

View the Dialogues in a Session

To list the dialogues in a session, run doctl signals session dialogues with a session ID from the session list:

doctl signals session dialogues --session-id <session-id>

Dialogues are listed in the order they occurred. The output shows each dialogue’s ID, its sequence in the session, its run status, the first 80 characters of the user’s message, and the types of any signals detected in that run.

To see the full user message, the steps taken to respond, and the details of each signal, add --output json:

doctl signals session dialogues --session-id <session-id> --output json

The JSON output contains one object for each dialogue:

[
  {
    "id": 1205818346715843946,
    "run_id": "5b4a3c2d-1e0f-4a9b-8c7d-6e5f4a3b2c1d",
    "created_at": "2026-09-30T19:42:03Z",
    "sequence": 2,
    "user_message": "Deploy the latest build to staging.",
    "steps": [
      {"seq": 1, "type": "agent_reply", "text": "I couldn't find a deployment tool."}
    ],
    "run_status": "completed",
    "failure_code": null,
    "signals": [
      {
        "signal_type": "ExecutionFailureToolNotFound",
        "label": "Tool not found",
        "run_uuid": "5b4a3c2d-1e0f-4a9b-8c7d-6e5f4a3b2c1d",
        "step_index": 1,
        "confidence": 0.92,
        "snippet": null,
        "layer": "execution",
        "category": "failure"
      }
    ]
  }
]

The main fields in each dialogue are:

Field Description
id The dialogue’s identifier. Some JSON parsers can’t represent large 64-bit integers exactly, so use a parser that preserves them.
run_id The identifier of the run that produced the dialogue.
sequence The dialogue’s position within the session.
user_message The message the user sent.
steps The steps taken to respond, in order.
run_status completed, incomplete, or failed.
signals The signals detected in this run. Each entry includes the signal type, its layer and category, and a confidence value from 0 to 1. The field is omitted when the run has no signals.

You can filter dialogues with --signal-type, --start-time, --end-time, and --limit, the same way as sessions.

Export Signal Data

Exports package your team’s Serverless Inference sessions, signals, and dialogues into a compressed JSON file that you can download for offline analysis. Exports run in the background, so you create an export, wait for it to complete, and then request a download URL.

Exports remove personal information from free text, such as user messages, step text, and signal snippets.

Create an Export

To create an export of all of your team’s Serverless Inference sessions, run:

doctl signals export create --type inference

To narrow the export, add --signal-type or --session-ids, both repeatable, or a time window with --start-time and --end-time in Unix epoch seconds. --start-time is inclusive and --end-time is exclusive. To list the signal types you can filter on, run doctl signals export options.

The signal type filter chooses which signals appear in the export. It does not remove sessions, so a session with no matching signals still appears with an empty signals list and its full dialogues.

If an identical export is already queued or running, the command returns that export instead of creating a new one.

The output shows the new export’s ID, status, signal type filter, and creation time. Export timestamps are Unix epoch seconds.

Check an Export’s Status

To check an export, run:

doctl signals export get <export-id>

The Status column has one of the following values:

Status Description
queued The export is waiting to be processed.
running The export file is being built.
complete The export file is ready to download.
failed The export failed. To see the reason, run the command again with --output json and check the error_message field.
expired The export file passed its 7-day retention period and can no longer be downloaded.

Small exports typically complete within seconds.

Download an Export

When the status is complete, request a download URL:

doctl signals export download <export-id>

The output shows a pre-signed download URL and the time it expires, in Unix epoch seconds. The URL is valid for 15 minutes. To get a new one, run the command again. The command returns a 404 error if the export is not complete yet, and a 410 error after the export expires.

To download and decompress the file, run the following commands, replacing <download-url> with the URL from the output:

curl -o signals-export.json.gz "<download-url>"
gunzip signals-export.json.gz

The file contains a nested JSON structure of sessions and their signals and dialogues, ordered from oldest to newest.

List Exports

To list your team’s exports, run doctl signals export list. The command returns 20 exports by default. Use --limit to return from 1 to 100.

Signals Data Deletion Request

Deleting Signals data permanently and irreversibly removes the conversation data and signal reports that Signals collected for the requested scope. Deletion cannot be undone. When you create a deletion, Signals immediately turns collection off for that scope, hides matching conversations from Signals results, then deletes that conversation data asynchronously in the background. You create a job, poll until it finishes, then check the result.

For Serverless Inference (--type inference), deletion is team-wide: Signals deletes Serverless Inference conversations for your whole team (not limited to one endpoint or model) that were collected at or before the deletion request, and turns off team inference collection. To resume collection, enable Signals for Serverless Inference again. You can do this before the deletion finishes. Do not pass --agent-id.

Deletion does not remove export files that you created earlier. They remain available to download until they expire (7 days).

Aggregated Insights or metrics charts may still show historical summary data points after deletion. Those summaries are not the same as recoverable conversation content.

Create a Deletion Request

To permanently delete the data, run doctl signals deletion create with the inference type:

doctl signals deletion create --type inference

doctl asks you to confirm before it sends the request. Add --force to skip the prompt, for example in a script.

The output shows the deletion ID, type, status, and creation time. Timestamps are Unix epoch seconds.

If a deletion for the same target is already queued or running, the command returns that job instead of creating a new one. Each team can have up to two deletions queued or running at a time. If you reach that limit, the command returns a 429 error. Wait for a job to finish, then try again.

Get Deletion Request Status

To check a deletion, run:

doctl signals deletion get <deletion-id>

The Status column has one of the following values:

Status Description
queued The deletion is waiting to start.
running Signals is deleting the data.
complete The data is permanently deleted.
failed The deletion failed. Matching conversations stay hidden and collection stays off. Run the command again with --output json and check error_message, then create the deletion again to retry.

Deletion runs asynchronously in the background. The time to complete depends on how much data needs to be removed. DigitalOcean may retry transient processing errors while a job is still active. If a job ends as failed, create the deletion again to retry.

List Deletion Requests

To list your team’s deletions, newest first, run:

doctl signals deletion list

The command returns 20 deletions by default. Use --limit to return from 1 to 100. If there are more results, doctl prints a next-page token. Pass it with --after <token> to fetch the next page.

Limits

  • Signals is in public preview, so its limits and behavior may change.
  • Accounts in a free trial can’t use Signals. Your team must also be in good billing standing.
  • Each team can have up to two deletions queued or running at a time.
  • Completed exports are available to download for 7 days.
  • List commands return up to 100 items.

We can't find any results for your search.

Try using different keywords or simplifying your search terms.