Serverless Inference Overview
What is serverless inference and how it differs from dedicated inference.
Inference How-Tos
Generated on 15 May 2026
Inference provides a single control plane for managing inference workflows. It includes a Model Catalog where you can view available foundation models, including both DigitalOcean-hosted and third-party commercial models, compare model capabilities and pricing, use routing to match inference requests to the best-fit model, and run inference using serverless or dedicated deployments.
Serverless Inference
Serverless Inference API Endpoints
Synchronous and asynchronous API endpoints for serverless inference.
Use Serverless Inference
Send API requests directly to foundation models without creating an AI agent or managing infrastructure.
How to Retrieve Available Models
How to retrieve models available for serverless inference.
How to Send Prompts to a Model Using the Chat Completions API
Send prompts and use reasoning with the Chat Completions API.
How to Send Prompts to a Model Using the Responses API
Send prompts with the Responses API.
How to Use Prompt Caching in Chat Completions and Responses API
Use prompt caching with the Chat Completions and Responses API.
How to Use Reasoning with the Chat Completions and Responses API
Use reasoning with the Chat Completions and Responses API.
How to Generate Images from Text Prompts
Generate or edit images from text prompts.
How to Use Multimodal Inference
Process and generate content across multiple data types, including images, audio, video, and text using multimodal models.
How to Use fal Models to Generate Image, Audio, or Text-to-Speech
Generate image, audio, or text-to-speech using fal models.
How to Convert Text Into Dense Vector Representations
Convert text into dense vector representations for use in semantic search, retrieval-augmented generation (RAG), clustering, classification, and similarity matching.
How to View Serverless Inference Metrics
View metrics such as latency, throughput, error rates, token consumption, cost attribution, and rate limiting.
How to Use Serverless Inference After Updating to Another Model
How to use serverless inference after updating a model.
Manage Model Catalog
How to Browse Models in Model Catalog
Identify the right model for your use case by filtering available foundation models by capabilities and price.
How to Import Your Own Models (BYOM)
Import Bring Your Own Models (BYOM) models into Model Catalog from Hugging Face or Spaces buckets and folders.
Test and Compare Models Using the Model Playground
Test and compare foundation models in the Model Playground.
Use Dedicated Inference
How to Use Dedicated Inference
Deploy open-source and commercial LLMs on dedicated GPUs as an inference endpoint.
Use Batch Inference
How to Use Batch Inference
Batch inference runs text jobs asynchronously through batch APIs compatible with OpenAI and Anthropic using your serverless inference model access key.
Use Inference Router
How to Use Inference Router
Create and configure an Inference Router to route inference requests to foundation models.
Agentic Workflows
How to Use Claude Code and Other Agentic Workflows on DigitalOcean
Use the Messages API with Claude Code and similar agentic workflows.
Evaluate Models
Use Built-in Tools
How to Use Built-in Tools
Extend model capabilities with server-side tools like knowledge base retrieval and MCP during inference requests with serverless and dedicated inference.
Use Agent Platform
Use Agent Platform
Build fully-managed AI agents with knowledge bases for retrieval-augmented generation, multi-agent routing, guardrails, and more.
How to Create Agents on DigitalOcean Inference
Create an agent with domain-specific knowledge to provide information or take action.
How to Build, Test, and Deploy Agents Using Agent Development Kit
Use Agent Development Kit to create and manage agents.
How to Add, Edit, or Delete Partner Provider Keys on DigitalOcean Inference
Add, edit, or delete API keys to use their models with your agents.
How to Create and Manage Workspaces on DigitalOcean Inference
Create workspaces to group agents together and move agents between workspaces as needed.
How to Use Agents in Your Applications on DigitalOcean Inference
Use your agent in an application or through a chat bot interface.
How to Test Agents Using the Agent Playground on DigitalOcean Inference
Test the full agent experience with the directions and features you’ve configured using the Agent Playground.
How to Evaluate Agent Performance on DigitalOcean Inference
Create test cases and measure how well your agents perform against for things like tone, factual accuracy, and context relevance.
How to Create an Evaluation Dataset on DigitalOcean Inference
Create effective evaluation datasets in CSV format to test your agent’s performance, improve accuracy, and measure qualities like factual correctness, safety, and instruction following.
How to View Agent Insights and Logs on DigitalOcean Inference
View insights and runtime logs for your agents to troubleshoot issues.
How to Trace Agent Responses on DigitalOcean Inference
Trace how your agent processes prompts to troubleshoot issues, improve performance, and control costs.
How to Route to Multiple Agents on DigitalOcean Inference
Integrate multiple generative AI agents.
How to Route Functions in Agents on DigitalOcean Inference
Enable the foundation model in your agent to access the external data sources using functions.
How to Rollback to a Previous Version of Agents on DigitalOcean Inference
Rollback to a previous version of an agent to undo changes made to it.
Create and Manage Knowledge Bases
Create, edit, manage data sources, verify, and permanently destroy knowledge bases.
Attach and Detach Agent Knowledge Bases
Attach or detach a knowledge base from your agents.
Manage Agent Guardrails
Create, manage, edit, duplicate, or delete guardrails to control how your agents respond to sensitive or inappropriate content.
How to Destroy Agents Using the Control Panel on DigitalOcean Inference
Destroy an agent to permanently and irreversibly destroy the agent and removes all endpoints for the agent.
Test DigitalOcean Knowledge Base Retrieval Using RAG Playground
Test how foundation models answer questions using content retrieved from a knowledge base.
Manage Model Access Keys
How to Create and Manage Model Access Keys
Create, scope, and manage model access keys for foundation models, inference routers, and batch inference, with VPC restrictions and team-owner visibility.