For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Vector search lets applications find data by meaning instead of only by exact keywords. It powers semantic search, recommendation systems, retrieval-augmented generation (RAG), image search, code search, and other AI applications that compare content by similarity.

DigitalOcean Vector Databases store vectors, index them for fast similarity search, and return the closest matches for a query vector. The same core concepts apply across all DigitalOcean Vector Database engines.

To choose the right engine for your application, see Choosing Between OpenSearch, Weaviate, and pgvector.

Vector Search Workflow

A typical vector search workflow has two phases:

1. Indexing: The application collects source content, converts each item into a vector with an embedding model, and stores the vector and metadata in the database. The database then indexes the vector for similarity search.
2. Querying: The application converts the user’s query into a vector with the same embedding model. The database searches for nearby vectors and returns the matching records, documents, or chunks.

We recommend using the same embedding model for indexing and querying. If you index documents with one model and embed queries with another, the vectors may not share the same dimensions or similarity behavior.

Vector Embeddings

A vector embedding is a list of floating-point numbers that represents a piece of content, such as text, an image, audio, or code. The embedding places that content in a vector space where distance represents similarity.

For example, two sentences with similar meanings produce embeddings that are close together. Two sentences with different meanings produce embeddings that are farther apart.

Embedding models create embeddings. These models are trained on large datasets to turn content into vectors that capture semantic meaning, structure, or similarity. Different models support different types of content:

• Text embeddings represent words, sentences, paragraphs, or documents.
• Code embeddings represent source code, functions, or documentation.
• Multimodal embeddings represent more than one content type, such as text and images in the same vector space.

Common embedding dimensions include 384, 512, 768, 1024, 1536, and 3072. A model’s output dimension is fixed. The vector field or column in your database must use the same dimension as the embedding model.

Changing embedding models usually means recomputing and reindexing your embeddings, because the new model may use a different dimension or organize similarity differently.

Where Embeddings Come From

Vector databases store and search embeddings. They don’t always create embeddings for you. Your application needs an embeddings workflow that turns source content into vectors before indexing, and then turns query text into a vector before searching.

There are two common approaches:

• Application-side embeddings: Your application calls an embedding model, receives the vector, and writes it to the database. This approach is portable across engines, easier to debug, and makes it easier to switch engines later because the embedding pipeline stays outside the database.
• Database-side embeddings: The database calls an embedding provider or model integration. This can simplify application code and let the database handle more of the ingestion workflow, but it adds more setup and engine-specific behavior. Weaviate supports built-in vectorization modules for providers such as OpenAI, Cohere, and Hugging Face. OpenSearch can use ML Commons for remote embedding models. Managed PostgreSQL typically generates embeddings in application code.

Prepare Data for Vector Search

Before you create embeddings, prepare your source data so search results are accurate, focused, and easy to filter. Most applications do this by splitting long content into smaller chunks and storing metadata with each embedding.

Chunking

For long documents, applications usually split content into smaller chunks before embedding it. Chunking helps because embedding models have context limits, and smaller chunks often produce more focused search results. A query usually matches a specific section of a document better than the entire document.

Chunk size affects search quality. Chunks that are too small may lose context. Chunks that are too large may include unrelated information and reduce relevance.

A good chunking strategy depends on the content type. Documentation, support articles, product descriptions, logs, and code may all need different chunking rules.

Metadata and Filtering

Most vector search applications store metadata alongside embeddings. Metadata describes the original item and lets you filter search results.

Common metadata fields include:

• Account ID
• Project ID
• Document type
• Timestamp
• Permissions
• Language
• Category
• Source URL

Filtering is important because semantic similarity alone is often not enough. For example, a search result may be semantically relevant but belong to the wrong customer, project, region, or permission scope.

Good vector database design keeps embeddings and metadata together so queries can combine similarity search with filters.

Measure Similarity

Before a vector database can return results, it needs a way to compare the query vector against stored vectors. Distance metrics define how similarity is measured, and nearest-neighbor search defines how the database finds the closest matches efficiently.

Distance Metrics

A distance metric measures similarity between two vectors. The embedding model determines which metric to use. Use the metric recommended by the model’s documentation.

Common distance metrics include:

• Cosine distance, which compares the angle between vectors and ignores magnitude. This is common for text embeddings.
• Inner product, also called dot product, which is often used when vectors are normalized or when a recommendation model is trained for this metric.
• Euclidean distance, also called L2 distance, which measures straight-line distance in vector space.
• Manhattan distance, also called L1 distance, which sums the absolute differences between vector values.

Cosine distance and inner product can produce the same ranking when vectors are L2-normalized. If your model outputs normalized vectors, inner product can be faster because the database does not need to normalize vectors at query time.

Exact and Approximate Nearest Neighbor Search

Vector search finds the nearest vectors to a query vector, called nearest neighbors. Exact nearest neighbor search compares the query vector against every vector in the dataset. This gives exact results, but it becomes slow as the dataset grows.

Approximate nearest neighbor (ANN) search uses an index to find close matches without comparing every vector. ANN indexes trade a small amount of recall for much faster queries, making them the standard approach for large vector datasets.

Exact search is useful when:

• Your dataset is small.
• A filter reduces the candidate set to a small number of vectors.
• You need exact ordering for validation or benchmarking.
• You re-rank a small set of ANN results.

HNSW Indexes

Hierarchical Navigable Small World (HNSW) is a common ANN index used for vector search. It organizes vectors into a graph that lets queries move quickly toward the closest matches.

HNSW uses multiple graph layers. Higher layers contain sparse, long-range connections that help the search move across the vector space quickly. Lower layers contain denser local connections that help refine the final results.

HNSW usually provides strong recall and low query latency, but it can use significant memory. Larger datasets, higher vector dimensions, and higher recall settings all increase memory usage.

Three common HNSW settings affect performance:

• m controls the number of graph connections per vector. Higher values can improve recall but use more memory.
• ef_construction controls how much work the database does when building the index. Higher values can improve index quality but increase build time.
• ef_search controls how many candidates the database considers at query time. Higher values can improve recall but increase query latency.

Hybrid Search

Hybrid search combines vector search and keyword search to improve relevance. It is useful when users need both semantic matches and exact matches in the same search experience. Vector search matches content with similar meaning, even when the content uses different words. Keyword search matches exact terms, such as product names, IDs, codes, rare terms, and specific phrases.

By combining both methods, hybrid search can return better results than vector search or keyword search alone.

Hybrid search is useful for:

• Product search
• Documentation search
• Support search
• Ecommerce search
• RAG applications over technical or structured content

The main challenge in hybrid search is score normalization. Keyword search and vector search produce different kinds of scores, so the database or application must normalize those scores before combining them.

DigitalOcean Vector Database engines support hybrid search in different ways:

• OpenSearch supports a native hybrid query with a normalization processor.
• Weaviate supports a built-in hybrid operator with controls for balancing keyword and vector relevance.
• PostgreSQL with pgvector supports hybrid search through application-level fusion, such as reciprocal rank fusion (RRF), over PostgreSQL full-text search and vector similarity results.

If you’re deciding which vector database to use, see Choosing Between OpenSearch, Weaviate, and pgvector. For engine-specific guidance, see the OpenSearch concepts and PostgreSQL concepts. To learn more about the HNSW indexing algorithm, refer to the HNSW paper by Malkov and Yashunin (2016).