For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Hybrid search combines keyword search and vector search in the same query so results can account for both exact matches and semantic similarity.

Because keyword and vector search use different scoring systems, OpenSearch normalizes each sub-query’s scores before combining them into one final document score.

How Score Normalization Works

Keyword search, such as BM25, scores documents based on term frequency, document length, and term rarity. These scores are unbounded and can vary widely between queries.

Vector search scores documents based on similarity or distance between vectors. These scores usually have a smaller and more predictable range.

OpenSearch normalizes scores so one query type does not dominate the final ranking only because it uses a larger score range.

Normalization Techniques

OpenSearch supports several normalization techniques for hybrid search:

• Use min_max for most workloads. It rescales each sub-query’s scores to a range between 0 and 1 based on the lowest and highest scores in the response. It is predictable, clear, and a good default when combining BM25 and k-NN scores.
• Use l2 only if benchmarks show that it performs better than min_max for your workload. L2 normalization rescales scores based on the magnitude of each sub-query’s score vector.
• Use z_score only when you understand your score distribution and have tested it with representative queries. Z-score normalization uses the mean and standard deviation of each sub-query’s scores, so outliers can affect ranking quality.

Combination Techniques

After normalizing scores, OpenSearch combines them into one final score. The combination technique controls how each sub-query contributes to the final ranking:

• Use arithmetic_mean for most workloads. It combines normalized scores using a weighted average and supports per-query weights. This makes it a good default when you want clear control over the balance between keyword and vector search.
• Use geometric_mean when a document should rank highly only if it performs well across multiple sub-queries. It can suppress documents that are strong in only one query type.
• Use harmonic_mean when low performance in any sub-query should strongly reduce the final rank. This is less common for general hybrid search because it can over-penalize useful results.

Choose Weights

Weights control how much each sub-query contributes to the final score. The order of weights must match the order of sub-queries in the hybrid query.

For a hybrid query with a BM25 sub-query followed by a k-NN sub-query:

• Use [0.5, 0.5] to give keyword and vector search equal weight.
• Use [0.7, 0.3] to favor keyword search.
• Use [0.3, 0.7] to favor vector search.

Start with equal weights, then tune the balance based on query type and relevance tests.

Increase the vector weight for longer, natural-language, or semantic queries. Increase the keyword weight for short, exact, or keyword-heavy queries, especially queries that include product names, commands, IDs, filenames, or error messages.

When testing weights, keep the normalization technique, combination technique, candidate count, and filters the same. This helps you tell whether ranking changes come from the weight change or from another part of the hybrid search configuration.

Test Relevance

The best normalization technique, combination technique, and weights depend on your data and queries.

Start with min_max normalization and arithmetic_mean combination because they are predictable, easy to tune, and provide clear control over the balance between keyword and vector scores.

Then, test different weights with representative queries and known relevant results.

Include different query types in the test set:

• Natural-language questions.
• Short keyword queries.
• Product names.
• Commands.
• Error messages.
• IDs or filenames.
• Queries with metadata filters.

Use the same test set when comparing configurations so you can tell whether a change improves relevance or only changes the result order.

Debug Score Changes

Use score explanations when results do not rank as expected.

When debugging score normalization, check:

• Whether each sub-query returns the expected candidates.
• Whether the weights match the sub-query order.
• Whether the normalization technique is appropriate for the score distribution.
• Whether the combination technique matches the ranking goal.
• Whether exact matches need stronger keyword weighting.
• Whether semantic matches need stronger vector weighting.

Make one tuning change at a time and compare results against the same test set.