For AI agents: The documentation index is at https://docs.digitalocean.com/llms.txt. Markdown versions of pages use the same URL with index.html.md in place of the HTML page (for example, append index.html.md to the directory path instead of opening the HTML document).

These benchmarks compare the small, medium, and large Managed Weaviate tiers on a 100,000-vector approximate nearest-neighbor (ANN) workload, with a separate run on a 1,000,000-vector workload for the large tier.

Methodology

What These Benchmarks Measure

Weaviate’s core operation is approximate nearest-neighbor search: given a query vector, return the k most similar vectors from the indexed set. Three metrics matter:

• Recall@10: Of the 10 results returned, what fraction are actually in the true top 10. 1.0 means perfect agreement with brute-force search.
• Throughput (QPS): Queries per second the cluster can sustain under concurrent load.
• Latency: Wall-clock time per query, reported as the mean and the 99th percentile (p99).

There is a fundamental tradeoff: the same index can be tuned to favor recall or speed, but not both at the same time.

The HNSW Index

Weaviate’s default vector index is HNSW (Hierarchical Navigable Small World graph). It has two phases:

• Build-time parameters, set when the index is created:
  • efConstruction: How thoroughly the graph is built. Higher values mean slower indexing and a higher recall ceiling. These benchmarks use 256 in all runs.
  • maxConnections (M): How many neighbors each graph node retains. Higher values mean more memory and slightly better recall. These benchmarks use 16 in all runs.
• Query-time parameter, swept across each benchmark:
  • ef: Search depth at query time. Higher ef explores more of the graph per query, which increases recall and latency and decreases QPS.

By holding efConstruction and maxConnections fixed and sweeping ef, each tier traces a recall-versus-throughput curve.

The dataset

Recall numbers from this dataset should generalize reasonably well to comparable semantic-search workloads.

The benchmark tool

These benchmarks use the open-source weaviate-benchmarking tool with the ann-benchmark subcommand. It performs:

1. Schema setup: Drops and recreates the Vector class with the configured HNSW parameters.
2. Ingest: Loads all vectors over gRPC using a producer-consumer pipeline with 8 worker goroutines and 100-vector batches.
3. Quiesce: Waits 30 seconds after ingest so the index can settle.
4. Query sweep: For each ef in {16, 24, 32, 48, 64, 96, 128, 256, 512}, runs the full test query set with 8 concurrent workers and records latency for every query.
5. Recall computation: Compares returned neighbors against the ground truth.

Test environment

• Load generator: A separate VM running the benchmarker CLI in the same region as the target cluster.
• Network: Traffic flows through the managed-cluster load balancer (TLS, gRPC over HTTPS on port 443). This adds non-trivial latency compared with localhost benchmarks. Keep this in mind when comparing absolute numbers to leaderboards that run in process.
• Concurrency: --parallel 8 for both ingest and query phases.
• Top-k: --limit 10. Recall and NDCG are reported at 10.

Small Tier (100k vectors)

Headline numbers

Full results

Observations

• Throughput does not decrease monotonically with ef. The expected ordering would be ef=16 > 24 > 32 > 48, but QPS jumps around. The most likely cause is single-run noise: each ef sweep runs for only a few seconds, so a brief CPU contention spike can dominate the measurement.
• p99 is consistently 4-8 times the mean, much higher than the medium and large tiers. This is the signature of a resource-constrained instance where slow queries queue behind faster ones.
• Import takes more than 10 minutes for 100,000 vectors, which is slow enough to matter for production data refresh patterns.

Recommended operating point

For a target of recall >= 0.98, run with ef=64:

If the application can tolerate ~95% recall in exchange for higher throughput, drop to ef=24 for ~159 QPS at 48 ms mean latency.

Suitability

The small tier is appropriate for development, staging, and low-traffic production workloads (up to about 100 sustained QPS). Its high tail latency makes it a poor choice for latency-sensitive interactive applications. For predictable sub-100 ms p99, use the medium or large tier.

Medium Tier (100k vectors)

Headline numbers

Full results

Observations

• The QPS curve is strongly non-monotonic, with throughput rising at ef=48 and ef=128 even though more work is being done. This is not algorithmically plausible. The most likely cause is noisy-neighbor variance on shared infrastructure during the brief window each ef runs.
• Tail latency ratios are healthier than the small tier (typically 2-7 times the mean), which suggests this tier has the headroom to absorb concurrent load.
• Import is about 3.4 times faster than the small tier on the same workload.

Recommended operating point

For recall >= 0.98, run with ef=64:

If a 99.5%+ recall target is required, ef=128 gave 348 QPS at 22 ms mean and 54 ms p99 in this run. The result is anomalously good and should be re-validated with a longer query duration before being relied on.

Suitability

The medium tier fits moderate-traffic production workloads with sustained throughput in the low hundreds of QPS at sub-100 ms p99. The unstable QPS curve in this single run means absolute numbers should be re-validated with longer duration sweeps before being quoted in external materials.

Large Tier (100k vectors)

Headline numbers

Full results

Observations

• Apart from a small QPS bump at ef=24 (typical of warmup effects in the first measurement window), the QPS curve is well-behaved and decreases monotonically.
• p99 stays under 30 ms for ef <= 128 and under 50 ms even at ef=512. The p99-to-mean ratio is consistently 2-3 times, which indicates plenty of CPU and memory headroom.
• Import takes 79 seconds for 100,000 vectors, which is fast enough to support frequent re-indexing or larger datasets.

Recommended operating point

For recall >= 0.98, run with ef=64:

For recall >= 0.99, step up to ef=128:

Suitability

The large tier handles production semantic-search workloads at scale: 1,000+ QPS with double-digit-millisecond p99 at high recall is suitable for latency-sensitive interactive applications such as chat, search, and recommendations. For 100,000-vector workloads the cluster shows substantial CPU and memory headroom, indicating that significantly larger datasets are viable on the same tier.

Tier Comparison

At the recall >= 0.98 operating point (ef=64) on the 100,000-vector workload:

The large tier delivered about 8.5 times the throughput of the small tier and about 3.4 times the throughput of the medium tier, with dramatically lower tail latency.

Iso-recall comparison

For each tier, the best operating point at common recall targets:

Recall >= 0.95

Recall >= 0.98

Recall >= 0.99

Recall >= 0.999

When to pick which tier

Bonus: Large Tier on a 1,000,000-Vector Workload

This run uses the 1,000,000-vector variant dbpedia-openai-1000k-angular.hdf5 (about 990,000 vectors after filtering) on the same large-tier hardware. It is excluded from the cross-tier comparison above because the dataset differs.

Headline numbers

Full results

Observations

• ef=16 is anomalously slow. The most likely cause is cold-cache effects on the very first iteration of the sweep. Treat that row with caution.
• From ef=24 onward the curve decreases roughly as expected with minor jitter, all within ~10-15%.
• Recall is lower at the same ef than on the 100,000-vector workload, which is expected. With 10 times the vectors, the HNSW graph is denser and the same ef explores a smaller fraction of it. To match a recall target on the 1,000,000-vector dataset, increase ef accordingly.
• p99 stays under 32 ms across the full sweep (excluding the anomalous ef=16), well within typical interactive-query SLAs for a million-vector index.
• Import time scaled slightly worse than linear: 944 seconds for 1,000,000 vectors versus 79 seconds for 100,000. HNSW build cost grows super-linearly with vector count.

Recommended operating point

For recall >= 0.98, run with ef=96:

For recall >= 0.99, use ef=256:

Same hardware, different scale

Side-by-side, large tier, both datasets, at recall >= 0.98:

A 10 times increase in dataset size reduced peak throughput by roughly 25-30% at comparable recall, which is a graceful degradation curve.

Caveats

1. Single-run measurements. Each tier was benchmarked once. Per-ef query sweeps run for only 2-4 seconds, so they are vulnerable to transient noise. The clearest example is the medium tier’s non-monotonic QPS curve. Re-run with --queryDuration 60 or longer per ef value to get statistically stable numbers before quoting any of these results in customer-facing material.
2. Noisy-neighbor variance. Managed multi-tenant infrastructure is subject to co-tenant interference. The unstable medium and small results may understate the steady-state performance of those tiers.
3. No memory metrics. Weaviate’s Prometheus endpoint is not exposed on managed clusters during preview, so heap and resident-set-size measurements are not available.
4. Single dataset, single dimensionality. All measurements are on 100,000 vectors at 1,536 dimensions (plus the bonus 1,000,000-vector run on the large tier). A smaller-dimension dataset would round out the picture.
5. Network latency floor. Traffic flows through the managed-cluster load balancer in TOR1, so absolute latencies include several milliseconds of networking overhead that would not be present in an in-process or same-VPC benchmark.