---
ai_assisted: true
title: '[PGConf.Asia 2026] A GPU Behind a PostgreSQL Index'
description: >-
  How pg_cuvs is built, what we measured, and what it costs. A written version
  of my PGConf.Asia 2026 talk, with the comparison against pgvector HNSW at
  recall 0.99 and its conditions.
pubDate: '2026-11-19'
category: DB
tags:
  - PostgreSQL
  - pgvector
  - pg_cuvs
  - cuVS
  - GPU
  - vector search
  - CAGRA
  - PGConf.Asia
  - Benchmark
heroImage: ''
draft: false
published: true
serial: 47
---

On November 18, 2026, I gave a 45-minute session at PGConf.Asia 2026 in Hong Kong: "pg_cuvs: GPU Vector Search as a PostgreSQL Index Access Method". pg_cuvs is an open-source PostgreSQL extension built on [pgvector](https://github.com/pgvector/pgvector) and [cuVS](https://github.com/NVIDIA/cuvs), NVIDIA's open-source library for GPU vector search, and it puts a GPU behind your vector index. This post is the talk in written form. The abstract and slides are on the [PGConf.Asia session page](https://pgconf.asia/EN/2026/proposals/?id=e6900fd2-1ccd-4735-9116-7bd5c8dda914), and the code is at [pg-cuvs/pg_cuvs](https://github.com/pg-cuvs/pg_cuvs). The abstract was written for the submission in September from earlier measurements, so its figures differ from the ones here. The numbers in this post are the November measurements shown in the talk.

The talk makes one claim. You can put a GPU behind a PostgreSQL index access method, and PostgreSQL still keeps the SQL, the planner and the transactions while the GPU only proposes candidates.

Here is the headline result with its conditions. On one million 768-dimensional vectors, with both sides tuned to a recall of 0.99 or higher, pg_cuvs on a 24-vCPU server with one A100 served about 6,520 queries per second at a 99th-percentile latency of 7.7 ms, with a 200 µs batching window turned on and 32 clients. The window is off by default, and without it the peak was about 1,560 queries per second. pgvector's HNSW on a separate 64-vCPU server peaked at about 3,550 queries per second with 64 clients, with a p99 of about 27 ms at that peak. In a second comparison against the GPU server's own 24 vCPUs (single runs), the GPU side served about three times as many queries at a recall of 0.99 or higher, the two were about equal near 0.95, and below that the CPU served more queries. These are our own measurements and have not been reproduced independently. The detailed conditions are listed below.

## The question

A graph index for vector search is allowed to miss rows, unlike a B-tree, because it looks at only part of the data. Recall measures how much it finds: nine of the true top ten is a recall of 0.9. You can always go faster by being a little less accurate, so every number in this post comes with its recall.

High recall is expensive. On a pgvector HNSW index over one million vectors, one query at a time, the median latency was about 1.2 ms at a recall of 0.91 and about 10.2 ms at a recall of 0.99 (pgvector 0.8.7, `m = 16`, `ef_construction = 256`, a 64-vCPU server, one client, single runs). Raising `hnsw.ef_search` makes the search keep more candidates, and every candidate costs a distance computation over 768 numbers. More cores serve more clients, but they do not make one query faster. A GPU can compute many of those distances at once.

People have put GPUs next to vector search before, in three broad ways. Some use the GPU to build the index and still search on the CPU. Some send the search to a separate GPU service, and then, as far as we can tell, visibility and permissions have to be kept in step across two systems. Some put the GPU inside a different database engine, and then you are no longer in PostgreSQL. Each gives something up.

pg_cuvs learns from all of them and asks one question: can the GPU do the search at query time while PostgreSQL stays the authority?

## How pg_cuvs is built

pg_cuvs is a PostgreSQL extension written in C and released under the PostgreSQL License. It is a GPU layer on top of pgvector's interface, not a replacement for pgvector. It uses pgvector's vector type and operators, and it leaves the GPU work to cuVS, whose graph index is called CAGRA. What we built is the part in between. pg_cuvs was also my experiment in building a database extension together with AI coding agents.

![Diagram of pg_cuvs. The extension loaded into each PostgreSQL backend talks to one GPU daemon over a Unix socket and shared memory, and the daemon loads the index files on disk into GPU memory once](https://media.jaesolshin.com/media/83d3ec8769c99f34c7ece6fe6f8ac67d36eb7db341977abdb98f4d2972e9d7e3.png)

*Figure 1. What pg_cuvs looks like. From the talk's slides.*

There are two pieces. The extension is loaded into every PostgreSQL backend and implements the index access method and the cost estimate. The GPU daemon, `pg_cuvs_server`, is a separate process that holds the GPU resources and a registry of the indexes. There is one daemon because PostgreSQL has one process per connection. If each backend used the GPU on its own, GPU memory would run out quickly, and the index should be loaded into GPU memory once and shared by every connection. Keeping a single process that owns the GPU is an idea from PG-Strom. Backends talk to the daemon over a Unix socket and shared memory.

An index access method is the interface through which PostgreSQL asks every kind of index the same few questions: can you build yourself, what would a scan cost, and what is the next row address. B-tree, GiST and pgvector answer on the CPU. pg_cuvs answers the search question with a GPU behind it. Because it plugs in at that point, the planner still picks the plan it estimates to be cheapest.

`CREATE INDEX` runs in four steps. The backend scans the table and collects pairs of a vector and its row address. It hands the vectors to the daemon as a shared memory file. cuVS builds the CAGRA graph in GPU memory. The daemon writes each index file to a temporary name, syncs it, renames it, and only then replies. If the build or the write fails, `CREATE INDEX` reports an error and the catalog change rolls back.

For a `SELECT`, the index scan sends one query vector to the daemon: 768 floats, about 3 KB. The GPU searches the graph over one million vectors, and the daemon returns 100 candidates as row addresses (TIDs) with their distances. 100 is the default of the `cuvs.k` setting. PostgreSQL reads those rows in order, checks each one against your snapshot and your WHERE clause, and stops when it has ten. The GPU proposes and PostgreSQL decides, so a row you are not allowed to see never reaches you. The other side of this is that PostgreSQL can only choose from what the daemon proposes, so the daemon's files have to stay in step with PostgreSQL. I come back to that below.

From the user's side, only the `USING` clause is new.

```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION pg_cuvs;

CREATE INDEX ON items USING cagra (embedding vector_l2_ops);

SELECT id FROM items ORDER BY embedding <-> $1 LIMIT 10;
```

The query does not change, so the application does not change. You install two things, the extension and the daemon, add pg_cuvs to `shared_preload_libraries`, and point PostgreSQL and the daemon at the same socket path and index directory. The project currently targets Linux and PostgreSQL 16.

There are four index types, and all of them use pgvector's `vector` type and operators.

| Type | Who answers the query | Notes |
|---|---|---|
| `cagra` | GPU | Graph index |
| `flat` | GPU | Compares the query against every vector, so it is exact |
| `ivfpq` | GPU | Compresses the vectors so more of them fit |
| `pg_cuvs_hnsw` | CPU (pgvector's own code) | The GPU only builds the graph, which is written in pgvector's HNSW page format. The daemon is needed only for the build |

## One query at a time

A GPU is used well when two things are true: the data is already next to it, and each call brings a lot of work. A database is the other way around on both counts. The data is large and lives in PostgreSQL's tables, and one query is a small job for a GPU. The daemon answers the first by keeping a copy of the index in GPU memory. The second was the center of the talk.

A PostgreSQL index scan carries one query vector, and each call returns one row address. A cuVS search call, in contrast, is designed to take a whole batch of queries. So every backend sends its own query, and the daemon makes one GPU call for each.

Under load it looks like this. On one CAGRA index at a recall above 0.99, throughput was about 1,180 queries per second with one client, about 1,550 with two, and still about 1,560 with four. At 64 clients it stayed at about 1,530. Meanwhile the server's 24 vCPUs were only about 6% busy, so the machine as a whole was not short of CPU.

The workaround in today's release is a short wait. For up to 200 µs, the daemon collects the searches that arrive for the same index and sends them to the GPU as one call. We call this the batching window, and you turn it on with the `cuvs.cagra_batch_wait_us` setting. The default is 0, which means off. With a 200 µs window the same index reached about 6,520 queries per second at 32 clients, about four times the throughput without the window. It has a price. With a single client there is nobody to wait for, so the window only adds delay, and throughput with one client dropped from about 1,180 to about 890 queries per second. Which default is right depends on the workload.

That ceiling without batching has to do with our own code. In the current release, the daemon holds one lock while a search runs on the GPU ([#233](https://github.com/pg-cuvs/pg_cuvs/issues/233)). So we built an experimental version that moves only the GPU call outside that lock, and measured it on a different A100 server with the same data.

![Throughput by number of clients. Release v0.9.2 without batching stays flat at about 1,530 queries per second, and the experimental build with the GPU call outside the lock reaches 9,743 queries per second at 64 clients](https://media.jaesolshin.com/media/cc61f53efb6f69aaf7c45ef4ab51b31524dc25adf8ceeab8edc93000a0b8245d.png)

*Figure 2. Throughput of the release and the experimental build. A second A100 server, one million vectors, three runs per point.*

With no batching and 64 clients, the release stayed at about 1,530 queries per second and the experimental build reached about 9,740. That is about six times the release, and higher than the release with the batching window on the same server (about 7,100 to 7,260 queries per second). At 64 clients, though, the release with the window had the lower p99, 11.6 ms against 14.6 ms for the experimental build. These numbers need care. The experimental build is not in a release, it is safe only while no index is dropped, rebuilt, updated or evicted from GPU memory during the run, and each point is three runs on one GPU. Still, the result points to our lock, not the GPU, as the ceiling. We have not yet measured batching together with this change.

## How we measured

The results below were measured under these conditions.

- Data: one million Wikipedia embeddings (wiki_all), 768 dimensions, L2 distance, `LIMIT 10`
- Accuracy: a recall of 0.99 or higher on both sides (0.993 for pgvector, 0.994 to 0.995 for CAGRA)
- Load: pgbench, prepared statements, closed loop, 60 seconds per client count, medians of five repeats
- GPU side: a 24-vCPU server with one A100 (80 GB), PostgreSQL 16.15, pg_cuvs 0.9.2, `cuvs.k = 100`
- CPU side: a separate 64-vCPU server (m8id.16xlarge), PostgreSQL 16.16, pgvector 0.8.7, HNSW with `m = 16`, `ef_construction = 200`, `hnsw.ef_search = 640`

We put the CPU side on a separate, larger server instead of the small CPU that comes inside a GPU machine. "pgvector at a recall of 0.99" is not one number, because it depends on the index settings, so the settings we used are listed above.

The slides also show two mistakes in our own benchmark setup. In the first, pgvector's parallel index build ran with 6 workers, not 63. The build on one million vectors came out at 180 seconds, and with the setting fixed it took 67. In the second, we set `hnsw.ef_search` above pgvector's limit of 1000. The setting fell back to its default of 40, and the 867 queries per second we measured with one client were at a recall of 0.90. In the formal run at a recall of 0.99 or higher, the figure was 87. Every number below comes after both fixes.

## Results

| | Peak throughput | Clients at the peak | p99 |
|---|---|---|---|
| pgvector HNSW (64-vCPU server) | 3,553 QPS | 64 | 26.7 ms |
| CAGRA, no batching | 1,561 QPS | 4 | 2.8 ms |
| CAGRA, 200 µs window | 6,520 QPS | 32 | 7.7 ms |

![Throughput by number of clients. pgvector HNSW peaks at 3,553 queries per second with 64 clients, CAGRA without batching stays flat at about 1,560, and CAGRA with a 200 µs window reaches 6,520 at 32 clients](https://media.jaesolshin.com/media/39a754225a17e55c2013630d61b121547195157de778e2265d34ec446b2545e6.png)

*Figure 3. Throughput by number of clients at a recall of 0.99 or higher, medians of five runs. pgvector ran on the 64-vCPU server and CAGRA on the 24-vCPU server with one A100.*

At the same recall, CAGRA with the 200 µs window, on a 24-vCPU server with one A100, served about 1.8 times as many queries per second as pgvector on the 64-vCPU server. The difference is clearer as a latency budget. If your budget is a p99 of 10 ms, pgvector with the index settings we measured has no point inside it: even a single client saw a p99 of about 17 ms.

The gap is largest for one query at a time. With a single client, the median latency was about 11.5 ms on pgvector and about 0.84 ms on CAGRA, so CAGRA's was about one fourteenth of pgvector's. With more clients the gap gets smaller.

Looking across recall shows where the GPU wins. The next comparison runs pgvector on the GPU server's own 24 vCPUs, with 64 clients. It is not the 64-vCPU server from the table, and these are single runs.

![Throughput across recall. CAGRA with a 200 µs window stays nearly constant at about 6,400 queries per second from a recall of 0.85 to 0.99, and pgvector HNSW falls from about 12,450 at 0.85 to 2,076 at 0.99. The two lines meet near 0.95](https://media.jaesolshin.com/media/fafcc9fc38a5d9706bffa92333eff24b5ee290367191f478708f498f0bb344fa.png)

*Figure 4. Best throughput at each recall or higher. 64 clients, pgvector on the same GPU server's 24 vCPUs, single runs.*

Each point is the best throughput among the settings that reach that recall or higher. At a recall of 0.99 or higher, CAGRA with the window served about 6,370 queries per second against about 2,080 for pgvector, about three times as many. At 0.95 or higher they were nearly equal, about 6,470 against about 6,490. At 0.90 or higher pgvector was ahead with about 10,390. pgvector gets much faster when you give up a little accuracy, and CAGRA did not. The GPU wins where you need very high accuracy.

Building the index on one million vectors took about 67 seconds for pgvector's HNSW (63 parallel workers, `m = 16`, `ef_construction = 200`), about 31 seconds for CAGRA, and about 27 seconds for the GPU-built HNSW (`pg_cuvs_hnsw`, `ef_construction = 128`). Each figure is the whole `CREATE INDEX` and the median of five runs. The two HNSW builds use different `ef_construction` values.

## What PostgreSQL's authority costs

Keeping PostgreSQL as the authority has a cost. Part of it came from PostgreSQL's interfaces, and much of it from our own code.

We ran the same settings twice, once calling cuVS directly and once through PostgreSQL. In all 315 pairs of settings, the path through PostgreSQL and our daemon added between 0.25 and 0.32 ms per query. That cost was nearly constant: it did not grow when the search got heavier. Each of the nine builds took about 15 seconds longer. These are single runs, the direct side is the mean latency at batch size 1, the PostgreSQL side is the median with one client, and the two sides used different libcuvs builds (public and patched).

We measured where those 15 seconds went on another server of the same type. Reading the table and handing it to the daemon took about 0.2 seconds. The remaining 15 or so seconds were the daemon saving its files, and more than half of that (about 8.3 seconds) was a checksum computed on one thread. Syncing the files to disk before replying took about 1.5 seconds.

The GPU-built HNSW told the same story. On a diagnostic build, a `CREATE INDEX` took about 82 seconds (one run), and about 49 of them were in our code that writes the graph into pgvector's index pages. After several fixes to that step, it takes about 26 seconds under the same conditions in the current release, and none of the changes touched the GPU work. This is a different measurement from the 27 seconds in the previous section. In both cases the cost was not in the GPU. It was in writing the result to disk and into PostgreSQL's pages.

There is one more cost, and it is on no chart. When an index lives in PostgreSQL's own pages, it follows PostgreSQL's commits, rollbacks and crash recovery. pg_cuvs's GPU indexes live in a separate process, so the daemon has to keep those promises itself. PostgreSQL can reject a wrong candidate, but it cannot notice a missing one. Shortly before the talk we found places where the current release does not keep that promise yet, and we filed them as public issues. For example, if the daemon dies between two file renames during a REINDEX, searches after the restart return wrong results without an error ([#235](https://github.com/pg-cuvs/pg_cuvs/issues/235)), and rows inserted through the extend path disappear from index searches after the daemon is killed ([#236](https://github.com/pg-cuvs/pg_cuvs/issues/236)). We have a new design for how the daemon saves and commits its files, but it is not in a release yet. If you plan to try the current release, read the [open issues](https://github.com/pg-cuvs/pg_cuvs/issues) first.

Here is what that cost buys. Your SQL does not change. The planner still chooses, and you can see its choice in `EXPLAIN`. In an index scan, visibility, filters, joins and permissions are still PostgreSQL's.

## Should you use it?

The talk reduced the decision to four questions.

1. Do you need very high accuracy, a recall of 0.99 or more? If not, tune pgvector first, and stay with it if it is enough. In the comparison against the GPU server's own 24 vCPUs, the CPU served more queries below a recall of about 0.95.
2. If you do, is your data bigger than GPU memory? Then consider the GPU-built HNSW: the build uses disk, and the CPU serves the queries. The disk route currently needs a patched cuVS and plenty of host memory (17 to 21 GiB at one million vectors).
3. If it fits, do you serve from the CPU and rebuild the index often? The GPU-built HNSW is again the candidate. It depends on pgvector's page format, so test it again after a pgvector upgrade.
4. Otherwise, consider CAGRA, and try the batching window if many clients arrive at once.

Whichever path you take, check the plan with `EXPLAIN` and measure with your own embeddings.

## What I took home from Hong Kong

One question in the Q&A asked how index updates work between the daemon and the backends. When a row is inserted into a CAGRA index, the daemon first tries to extend the graph on the GPU. If that fails, the backend appends the row address and the vector to a `.delta` file, and a search merges the candidates from the graph with the result of searching that delta.

I also made one request to the community. If an index access method could take many query vectors at once, a query that carries many vectors, such as a join, could be sent to the GPU as one batch without waiting. Today that needs a separate function, `pg_cuvs_batch_search`, and it changes your SQL.

The last part of the talk was about what this project taught me. I have been absorbed in one idea, PostgreSQL with a GPU, and because of that my imagination about who needs it, and for what, only went so far. The design keeps arriving at the same kind of choice. Should we tune for the lowest latency of one query, or for throughput under load? Should one daemon serve every connection, or should the GPU work happen inside PostgreSQL's own processes? Should the daemon keep the index in step with PostgreSQL through its own files, or should the index live in PostgreSQL's pages and its WAL? When the data outgrows one GPU, should we stay on GPUs, or build on the GPU and serve from the CPU? Our measurements tell us what each side gives. They do not tell us which side matters more. Only a real workload, and the person who owns it, can do that. What I learned is that a project like this needs partners on the demand side to think these trade-offs through with.

This talk also connected me with the PostgreSQL contributor community, and I am glad to have started talking with people there.

If PostgreSQL with a GPU would help your application, I would like to hear what you need. [GitHub issues](https://github.com/pg-cuvs/pg_cuvs/issues) and pull requests are welcome.

## Thanks

Thanks to WeDataLab, to Corey Nolet at NVIDIA for the time and feedback, and to Steve Choi and Yujin Oh.
