For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

Managed Weaviate performance varies by tier, dataset size, and query-time tuning. The comparison below shows how the small, medium, and large tiers behave under a 100,000-vector approximate nearest-neighbor (ANN) workload, with an additional 1,000,000-vector run on the large tier to illustrate how performance scales.

Methodology

What These Benchmarks Measure

Weaviate’s core operation is approximate nearest-neighbor search: given a query vector, return the k most similar vectors from the indexed set.

The benchmarks report four metrics:

• Recall@10: Fraction of returned results that are in the true top 10. 1.0 means perfect agreement with brute-force search.
• Throughput (QPS): Queries per second under concurrent load.
• Latency: Wall-clock time per query, reported as mean and p99.
• NDCG@10: Normalized discounted cumulative gain at rank 10. It scores how highly relevant results are ranked within the top 10, with more weight on positions closer to the top. Reported in the per-ef tables next to Recall@10.

There is a fundamental tradeoff: the same index can be tuned to favor recall or speed, but not both simultaneously.

The HNSW Index

Weaviate uses the HNSW (Hierarchical Navigable Small World) index.

Build-time parameters (fixed in these benchmarks):

• efConstruction = 256: Higher values improve recall at the cost of slower indexing.
• maxConnections (M) = 16: Controls graph density and memory usage.

Query-time parameter (swept):

• ef: Search depth. Higher values increase recall and latency, and reduce QPS.

At index build, efConstruction and M are fixed, and only ef is swept at query time to trace a recall-throughput curve for each tier.

The Dataset

Results generalize reasonably well to similar semantic search workloads.

Benchmark Tool

These benchmarks use the open-source weaviate-benchmarking tool with the ann-benchmark subcommand. It performs:

1. Schema setup: Drops and recreates the Vector class with the configured HNSW parameters.
2. Ingest: Loads all vectors over gRPC using a producer-consumer pipeline with 8 worker goroutines and 100-vector batches.
3. Quiesce: Waits 30 seconds after ingest so the index can settle.
4. Query sweep: For each ef in {16, 24, 32, 48, 64, 96, 128, 256, 512}, runs the full test query set with 8 concurrent workers and records latency for every query.
5. Recall computation: Compares returned neighbors against the ground truth.

Test Environment

• Load generator: A separate VM running the benchmarker CLI in the same region as the target cluster.
• Network: Traffic flows through the managed-cluster load balancer (TLS, gRPC over HTTPS on port 443). That path adds non-trivial latency compared with localhost benchmarks, so absolute numbers can look slower than leaderboards that run in process.
• Concurrency: --parallel 8 for both ingest and query phases.
• Top-k: --limit 10. Recall and NDCG are reported at 10.

Small Tier (100k Vectors)

Headline Numbers

Full Results

Observations

• Throughput does not decrease monotonically with ef. The expected ordering would be ef=16 > 24 > 32 > 48, but QPS jumps around. The most likely cause is single-run noise: each ef sweep runs for only a few seconds, so a brief CPU contention spike can dominate the measurement.
• p99 is consistently 4-8 times the mean, much higher than the medium and large tiers. This is the signature of a resource-constrained instance where slow queries queue behind faster ones.
• Import takes more than 10 minutes for 100,000 vectors, which is slow enough to matter for production data refresh patterns.

Recommended Operating Point

For most workloads targeting high recall, ef=64 provides the best balance:

If lower recall (~95%) is acceptable, ef=24 increases throughput to ~159 QPS.

Suitability

Best for development, staging, and low-traffic production (about 100 QPS). High tail latency makes it a poor fit for latency-sensitive applications.

Medium Tier (100k Vectors)

Headline Numbers

Full Results

Observations

• The QPS curve is strongly non-monotonic, with throughput rising at ef=48 and ef=128 even though more work is being done. This is not algorithmically plausible. The most likely cause is noisy-neighbor variance on shared infrastructure during the brief window each ef runs.
• Tail latency ratios are healthier than the small tier (typically 2-7 times the mean), which suggests this tier has the headroom to absorb concurrent load.
• Import is about 3.4 times faster than the small tier on the same workload.

Recommended Operating Point

For recall >= 0.98, ef=64 offers a strong balance:

If a 99.5%+ recall target is required, ef=128 gave 348 QPS at 22 ms mean and 54 ms p99 in this run. The result is anomalously good and should be re-validated with a longer query duration before being relied on.

Suitability

The medium tier fits moderate-traffic production workloads with sustained throughput in the low hundreds of QPS at sub-100 ms p99. The unstable QPS curve in this single run means absolute numbers should be re-validated with longer duration sweeps before being quoted in external materials.

Large Tier (100k Vectors)

Headline Numbers

Full Results

Observations

• Apart from a small QPS bump at ef=24 (typical of warmup effects in the first measurement window), the QPS curve is well-behaved and decreases monotonically.
• p99 latency remains under 30 ms for ef <= 128 and under 50 ms at ef=512. The p99-to-mean ratio is consistently 2-3 times, indicating available CPU and memory headroom.
• Import takes 79 seconds for 100,000 vectors, which is fast enough to support frequent re-indexing or larger datasets.

Recommended Operating Points

For recall >= 0.98, run with ef=64:

For recall >= 0.99, step up to ef=128:

Suitability

On the 100,000-vector workload in this benchmark, the large tier’s per-ef table shows higher QPS and lower mean and p99 latencies than the small and medium tiers for most settings. At recall >= 0.98 with ef=64, the operating-point table lists 1,109 QPS, 7 ms mean, and 15 ms p99, with 79 s import time for that dataset. The 1,000,000-vector section below uses the same large-tier SKU for a follow-up run.

Tier Comparison

At recall >= 0.98 (ef=64):

In the tier comparison table above, each larger tier shows higher QPS, lower mean and p99 latency, and shorter import time than the tier above it. The steepest p99 improvement is between small and medium; the largest QPS gain is between medium and large (see that table for exact values).

Iso-Recall Comparison

For each tier, the best operating point at common recall targets:

Recall >= 0.95

Recall >= 0.98

Recall >= 0.99

Recall >= 0.999

When to Pick Each Tier

Large Tier (1,000,000 Vectors)

This run uses the 1,000,000-vector variant dbpedia-openai-1000k-angular.hdf5 (about 990,000 vectors after filtering) on the same large-tier hardware. It is excluded from the cross-tier comparison above because the dataset differs.

Headline Numbers

Full Results

Observations

• ef=16 is anomalously slow. The most likely cause is cold-cache effects on the very first iteration of the sweep. Treat that row with caution.
• From ef=24 onward the curve decreases roughly as expected with minor jitter, all within about 10-15%.
• Recall is lower at the same ef than on the 100,000-vector workload, which is expected. With 10 times the vectors, the HNSW graph is denser and the same ef explores a smaller fraction of it. To match a recall target on the 1,000,000-vector dataset, increase ef accordingly.
• p99 stays under 32 ms across the full sweep (excluding the anomalous ef=16), well within typical interactive-query SLAs for a million-vector index.
• Import time scaled slightly worse than linear: 944 seconds for 1,000,000 vectors versus 79 seconds for 100,000. HNSW build cost grows super-linearly with vector count.

Key Insight

Increasing dataset size by 10 times reduced throughput by about 25-30% at comparable recall, indicating relatively stable performance on the large tier.

Recommended Operating Points

For recall >= 0.98, run with ef=96:

For recall >= 0.99, use ef=256:

Same Hardware, Different Scale

Side-by-side, large tier, both datasets, at recall >= 0.98:

Notes on Results

• Single-run measurements: Each tier was benchmarked once. Per-ef query sweeps run for only 2-4 seconds, so they are vulnerable to transient noise. The clearest example is the medium tier’s non-monotonic QPS curve. Re-run with --queryDuration 60 or longer per ef value to get statistically stable numbers before quoting any of these results in customer-facing material.
• Noisy-neighbor variance: Managed multi-tenant infrastructure is subject to co-tenant interference. The unstable medium and small results may understate the steady-state performance of those tiers.
• No memory metrics: Weaviate’s Prometheus endpoint is not exposed on managed clusters during preview, so heap and resident-set-size measurements are not available.
• Single dataset and dimensionality: All measurements are on 100,000 vectors at 1,536 dimensions (plus the 1,000,000-vector run on the large tier). A smaller-dimension dataset would round out the picture.
• Network latency floor: Traffic flows through the managed-cluster load balancer in TOR1, so absolute latencies include several milliseconds of networking overhead that would not be present in an in-process or same-VPC benchmark.