---
ai_assisted: true
title: What Happens to HNSW Recall After Adding Vectors? (10K+10K)
description: >-
  A small pgvector and Qdrant experiment compares recall before and after vector insertion, then against fresh indexes over the same final data.
pubDate: 2026-10-28T00:00:00.000Z
category: DB
tags:
  - PostgreSQL
  - pgvector
  - Qdrant
  - HNSW
  - vector database
  - benchmark
heroImage: ''
draft: false
published: true
---

The follow-up [1M+1M vector experiment](https://jaesolshin.com/posts/hnsw-incremental-insertions-1m/) scales this test up and examines insertion behavior using segment telemetry and source code.

A vector search index rarely stays exactly as it was built. New documents arrive, sometimes in groups about the same topic. It is straightforward to check that a newly inserted vector can be found. It is harder to tell whether the resulting index searches as well as an index built from scratch over the same data.

I tested that question on a small scale with pgvector and Qdrant. Each run started with **10,000 public embeddings and added another 10,000**. I varied the order of the new vectors and whether they arrived in one logical batch or five, then compared the resulting index with fresh builds over the same final vectors.

Across the same 200 queries, **Recall@100 was lower after insertion in all 48 histories**. The dataset and its true neighbors changed as well, so that drop cannot all be attributed to index degradation. Against separately built fresh indexes over the same final data, the incremental gaps ranged from −0.0822 to +0.1128 percentage points.

## Why compare against a fresh index?

HNSW finds approximate nearest neighbors by traversing a graph of vector connections. It can return results faster than an exact scan, but it may miss some true neighbors. Recall@100 measures the fraction of an exact search's top 100 neighbors that appear in the returned top 100.

A before-and-after recall comparison mixes two changes. Once the collection grows from 10,000 to 20,000 vectors, the nearest neighbors themselves can change. A difference in recall could reflect the new dataset, the index construction history, or both.

For each measured state, I computed exact neighbors from its current vectors and compared the incrementally updated index with an index freshly built over that same state. At the final state, three indexes were built independently in separate database instances; their per-query recalls were averaged before comparison. This did not measure the total cost of inserting into an existing database and then rebuilding its index:

```text
history gap = recall of incremental index − mean recall of fresh builds
```

A negative gap means the incremental index scored lower than the fresh-build average for those queries; a positive gap means it scored higher. Fresh builds vary too, so the gap alone cannot explain why the graphs differ.

## Fresh builds and four insertion paths

The data came from two public 768-dimensional embedding sets: MS MARCO Web Search SimANS and Wikipedia–Cohere. Search used the inner product of the original vectors. For each dataset, three seeds selected 20,000 vectors, split into an initial set and an incoming set of 10,000 each. The seeds share some documents and use the same evaluation queries.

I created two insertion orders for the incoming set. The mixed order shuffles all new vectors. The clustered order groups vectors using spherical k-means with five clusters, then inserts each cluster contiguously. Clustering uses normalized copies to group vectors by direction; the vectors used for search remain unchanged, and search uses their original inner products.

| Path | Order of incoming vectors | Logical batches |
|---|---|---:|
| M1 | Mixed | 1 |
| M5 | Same order as M1 | 5 |
| C1 | Grouped by cluster | 1 |
| C5 | Same order as C1 | 5 |

M5 uses the same batch sizes as C5. Within each dataset, database, and seed, all four paths start from the same physical initial index snapshot and reach the same final set. Vectors within each cluster are shuffled, not sorted by distance.

A logical batch is not one network request: transport chunks contained at most 1,000 vectors. Changing the number of logical batches also changes commits, waiting, and intermediate measurements. Qdrant can perform background optimization between requests.

![Mixed and clustered insertion paths with one or five logical batches](https://media.jaesolshin.com/media/c999fcc2f378a1f22d7533403b259bdbb014711177565174c8458dd2600ae06c.png)

*Figure 1. All paths start from the same initial index and reach the same final dataset. Colors illustrate clusters; actual cluster sizes differ.*

## What changed before and after insertion?

I compared Recall@100 for the **same 200 queries** on the initial 10,000 vectors and on the final 20,000. Exact neighbors were computed separately for each state. Final minus initial was negative in all 48 histories, ranging from **−0.8450 to −0.0700 percentage points**.

![Change in Recall@100 for the same queries before and after insertion](https://media.jaesolshin.com/media/ffac5ed1b913c1292812a45d2129f4b91e8630ffa432a222d1c206a1641ae7b6.png)

*Figure 2. Each point is final minus initial Recall@100 for one insertion path and seed. Blue denotes mixed order (M), and brown denotes clustered order (C). An x-axis value of −0.8 means a decrease of 0.8 percentage points. The same 200 queries are used at both stages, but exact neighbors are computed from the data present at each stage. No confidence intervals are shown.*

Adding vectors can change the true neighbors and the difficulty of a query. The drop is a measured before-and-after change, but it does not by itself establish that the HNSW graph degraded. To examine the construction history, I compared indexes over the same final data.

## Fresh, one batch, and five batches on the same data

For the final 20,000 vectors I used the **same 600 queries**. F is the mean recall of three independently constructed fresh indexes in separate databases. Each table entry is a path's Recall@100 minus F, shown as the range across three seeds for that dataset and database. Negative values mean the incremental index scored lower than F. Values are in percentage points.

| Dataset | DB | Mixed 1 − F | Mixed 5 − F | Clustered 1 − F | Clustered 5 − F |
|---|---|---:|---:|---:|---:|
| MS MARCO | pgvector | -0.0211~+0.0283 | -0.0144~+0.0300 | -0.0028~+0.0600 | -0.0061~+0.0617 |
| MS MARCO | Qdrant | -0.0789~+0.0611 | -0.0339~+0.1128 | -0.0622~+0.1061 | -0.0822~+0.0594 |
| Wikipedia–Cohere | pgvector | -0.0417~+0.0028 | -0.0450~+0.0028 | -0.0150~+0.0161 | -0.0150~+0.0161 |
| Wikipedia–Cohere | Qdrant | -0.0167~+0.0367 | -0.0100~+0.0517 | -0.0033~+0.0628 | -0.0033~+0.0461 |

![Recall difference from fresh builds across all four incremental paths](https://media.jaesolshin.com/media/3a7feb3e0c80a55af40c4d1f74bcf73b5b29b3f02574880a30c5293d9e658650.png)

*Figure 3. Blue denotes mixed order (M), and brown denotes clustered order (C). Points are incremental−F gaps for each path and seed. Bars are descriptive 95% paired-query bootstrap intervals; they do not capture all build variability and are not adjusted for multiple comparisons.*

Across all 48 histories, incremental−F point estimates ranged from **−0.0822 to +0.1128 percentage points**. This is a range across conditions, not a confidence interval. Of the 48 descriptive 95% intervals, nine were entirely positive and 39 included zero. These unadjusted intervals do not establish a product-wide advantage.

The 24 descriptive 95% intervals for five batches minus one batch under each order all included zero. The original prespecified primary comparisons were different: **C5−M5 and the order-by-splitting interaction**, 24 comparisons whose multiple-comparison-adjusted intervals all included zero. These intervals resample queries against completed, fixed indexes. Neither result establishes equivalence between paths or an advantage for smaller batches.

The before-and-after comparison uses 200 queries; the final fresh comparison uses 600. They answer different questions, so their values should not be subtracted to produce an estimate of index degradation.

## Comparing ef on the final indexes

Before the main experiment, separate calibration documents and queries were used to select HNSW build settings and search `ef`. Eligible settings reached at least 99% for both Recall@10 and Recall@100 across five calibration states; selection also considered latency, build and write cost, and memory. The settings were not retuned on the evaluation results.

The selected search ef was 1000 for MS MARCO with pgvector, 512 for Wikipedia–Cohere with pgvector, and 256 for Qdrant on both datasets. Passing the calibration threshold did not guarantee the same recall on other queries: two MS MARCO/Qdrant C5 seeds fell below 99% in evaluation.

I also queried the final C5 indexes at ef=100 and 128. For MS MARCO, the table shows the range across three seeds on the final set of 600 evaluation queries.

| Search ef | pgvector Recall@100 | Qdrant Recall@100 |
|---:|---:|---:|
| 100 | 83.58–84.15% | 92.80–93.36% |
| 128 | 87.33–87.66% | 95.20–95.59% |
| Selected operating ef | 99.70–99.80% · ef=1000 | 98.92–99.00% · ef=256 |

![Recall at different search ef values on the same final C5 indexes](https://media.jaesolshin.com/media/1b14142f47b66aab1d3991d73a76ee615ae49b9c9390ebc5630d7cce27d46968.png)

*Figure 4. Points are measured at ef=100, 128, and the selected operating ef for each combination. Connecting lines do not represent measurements at intermediate ef values.*

Qdrant achieved higher recall at the same numerical ef values of 100 and 128. However, pgvector used `M=16, ef_construction=100` and Qdrant used `M=48, ef_construction=300`; the same numerical ef does not imply equal search cost across implementations. The table is therefore not a comparison of product speed or efficiency.

The additional ef measurements were taken only at the final state. Without corresponding initial-state results at the same ef, this table cannot explain why Qdrant's before-and-after decline was larger than pgvector's.

## Was the search actually using HNSW?

A high recall obtained through an exact or sequential scan would not answer the index question. For PostgreSQL, the initial bulk load used `COPY`, followed by index creation; new rows were added with the index in place. The search session set `enable_seqscan=off`, and each query's plan was checked for the designated HNSW index. Representative queries were also checked with `EXPLAIN ANALYZE`.

Qdrant received new IDs in requests of at most 1,000 points. Since a successful write does not by itself establish that indexing is complete, each measurement waited for the data and indexing state to settle. Search telemetry was checked across the timed pass: HNSW counts had to increase while plain and exact search counts did not.

These checks apply to the evaluation searches in this experiment. They do not control every aspect of Qdrant's background optimization or every production workload. The databases ran sequentially on a shared local host, with a limit of two CPUs and 4 GiB of memory per database.

## What about larger indexes?

Other studies have observed larger differences under different update patterns. An [Azure Cosmos DB DiskANN paper](https://www.vldb.org/pvldb/vol18/p5166-upreti.pdf) used a 10-million-vector dataset with clustered insertion and deletion. A policy that repairs connections around deletions achieved up to 20 percentage points higher recall than a policy that simply drops deleted vectors in that clustered workload. That is a comparison of **deletion policies**, not the before-and-after effect of insertion alone.

The [IP-DiskANN study](https://arxiv.org/html/2502.13826v1) compared HNSW and DiskANN-family algorithms under a 30-million-vector clustered workload and rebuilt an index over the active data at selected checkpoints. Its algorithm results cannot be transferred directly to pgvector or Qdrant, but they give a reason to test larger datasets and additional update patterns.

The fresh control was built separately over the final data. The total time and cost of inserting into an existing database and then rebuilding its index were not measured.

This pilot measured **new-ID insertion**. It did not test deletion, repeated delete-and-insert cycles, or vector replacement under the same ID. Filters, selectivity, metadata cardinality, larger datasets, and long-running updates remain follow-up work. Once resources are available, those tests can retain the same-data fresh-build comparison.

The [full technical report and experiment results](https://github.com/ysys143/incremental_vector_upsert/blob/main/docs/real-vectors/clustered/technical-report.md) contain the configurations, complete tables, statistical intervals, and links to the published aggregates.