Best Vector Database for RAG: How to Choose for Production

The best vector database for RAG is usually the one your team already runs, until filtering, tenancy, or query volume force a move. If you run Postgres and have a few million chunks, start with pgvector. If exact codes matter, pick an engine with native BM25. If you serve many customers, price the index layout before the vendor.

Docs and prices below are current as of October 15, 2026. The surprise in the cost math: how you split the index changed the read bill by 100x. The product choice changed it far less.

Best vector database for RAG: a decision table by situation

“Start with” means the first candidate I would put into a bake-off, not a final answer. The table is my judgment from the vendor docs cited below. It is not a ranking.

If two rows fit, the stricter constraint wins. Residency beats convenience. Tenancy beats raw speed. And a vector database is one part of retrieval. Chunking, metadata, and evaluation usually decide more of your answer quality than the engine.

When is pgvector enough for production RAG?

pgvector is enough when your latency target holds at your real index size and query rate. HNSW, the graph index most of these systems use, hops between neighbor nodes on every query. The pgvector FAQ says indexes do not need to fit in memory, but “you’ll likely see better performance if they do.” When nodes sit on disk, hops become disk reads, so benchmark p95 latency and disk reads at full size.

So do the byte math first. The pgvector README (v0.8.7) says each vector takes 4 * dimensions + 8 bytes, and each halfvec takes 2 * dimensions + 8. At 1,536 dimensions, that is 6,152 bytes per full vector, or 61.5 GB for 10 million chunks before any graph overhead. Half precision cuts it to 3,080 bytes, or 30.8 GB. Indexed vector columns cap at 2,000 dimensions. Above that, the README lists half precision (up to 4,000), binary quantization (up to 64,000), indexing subvectors, or dimensionality reduction.

The best public pgvector numbers I found are Supabase’s sizing benchmark. On 1 million 1,536-dimension vectors at 0.99 accuracy, its 2XL instance (32 GB RAM) served 560 queries per second at 58 ms p95. Its 4XL (64 GB) served 950 at 33 ms p95. The table stops at 1 million vectors for that dimension, so my 10 million sizing below is an estimate.

Decision rule (mine): if the halfvec index plus your hot rows fit in RAM with about 30% headroom, and you want permissions joined in SQL, start with pgvector. If they do not fit, measure before you rule it out. Postgres extensions such as pgvectorscale and VectorChord add disk-friendlier indexes. I have not benchmarked them here.

Filtering and multi-tenancy are where production RAG breaks

Real RAG queries are rarely “nearest neighbors in the whole corpus.” They are “nearest neighbors this user may see, in this customer’s documents.” That filter is where recall quietly collapses.

The mechanism is post-filtering. Per the pgvector README, with approximate indexes, “filtering is applied after the index is scanned.” With the default hnsw.ef_search of 40 and a filter matching 10% of rows, you get about 4 results on average. The fix since v0.8.0 is iterative index scans (hnsw.iterative_scan), which keep scanning until enough rows match, up to hnsw.max_scan_tuples (20,000 by default). Turn it on and measure.

Tenancy is the same problem at a larger scale. Qdrant’s multitenancy guide advises against a collection per tenant in most cases. It recommends a payload field marked is_tenant, and since v1.16, promoting large tenants to their own shard at about 20,000 points. Weaviate stores each tenant on its own shard and lets you set tenants to ACTIVE, INACTIVE, or OFFLOADED to cloud storage. Pinecone uses namespaces. S3 Vectors allows 10,000 indexes per vector bucket.

My rule: physical separation (a namespace, shard, or index per tenant) beats a WHERE tenant_id = ? clause. With a filter, one missing clause leaks data across customers. With separation, a leak needs the wrong namespace name.

Early detection: for each filtered query, compare the result count against the smaller of k and the number of indexed records that pass the filter. A filter matching 3 records can only return 3. Returning fewer means the filter is starving the index.

Do you need hybrid search (BM25 plus vectors)?

You need hybrid search when users type things embeddings blur. As an illustration (I did not run a similarity test), part numbers like “AX-4410” and “AX-4401” can land close together in embedding space. BM25, the classic keyword ranking, rewards rare exact terms because they appear in few documents. Hybrid search runs both and merges the lists, often with Reciprocal Rank Fusion (RRF), which uses rank positions.

The best public evidence I know is Anthropic’s contextual retrieval study (September 2024). Its metric was the share of relevant chunks missing from the top 20. Contextual embeddings alone cut that failure rate from 5.7% to 3.7%. Adding contextual BM25 cut it to 2.9%, and adding a reranker cut it to 1.9%. BM25 sat on top of contextual embeddings there, and your corpus will differ.

Support varies. Elasticsearch and OpenSearch rank text with BM25 by default and add vectors. Weaviate, Qdrant, turbopuffer, and MongoDB Atlas offer keyword and vector search together. pgvector pairs with Postgres full-text search plus RRF, but Postgres’s built-in text ranking is not BM25. S3 Vectors has no keyword search. AWS points you to OpenSearch when you need hybrid search.

Decision rule (mine): if more than about 1 in 10 of your real queries contain an identifier, a code, or an exact name, require hybrid search in the bake-off. Keep it only if it measurably raises recall on those queries.

Latency targets, update rate, and data residency

Latency. Object-storage engines trade cold latency for cheap storage. turbopuffer’s architecture page reports p50 latency on 1 million documents of 874 ms for a cold query and 14 ms once the namespace is cached on SSD. AWS describes S3 Vectors as “subsecond” for infrequent queries and “as low as 100 milliseconds” for frequent ones. If a quiet tenant needs a 200 ms answer, test its first query of the day.

Update rate. S3 Vectors caps each index at 2,500 inserted or deleted vectors per second. Re-embedding 10 million chunks into one index takes at least 10,000,000 / 2,500 = 4,000 seconds, about 67 minutes. turbopuffer commits one write-ahead log entry per second per namespace and indexes in the background. In pgvector, churn adds HNSW inserts and vacuum work to the box serving queries.

An easy write event to miss is changing embedding models. Every vector gets rebuilt, so plan it as a migration with both indexes live during cutover.

Residency. Self-hosted options keep everything inside your network. turbopuffer offers single-tenant clusters and bring-your-own-cloud on its enterprise tier. If your vectors come from regulated documents, my view is to treat them as the same data class as the source, because they are derived from it. See data sovereignty for AI agents and private agents in your own cloud for boundary design.

What does a vector database cost? A worked estimate for 10M chunks

Hypothetical corpus, with my assumptions. Swap in yours:

• 10 million chunks, 1,536-dimension float32 embeddings.
• Per record: 6,144 bytes of vector, 1,000 bytes of filterable metadata, a 56-byte ID. That totals 7,200 bytes.
• Chunk text lives in your primary store. The app runs in the same cloud region.
• 1 million queries a month (0.39 per second on average).
• 10% of chunks re-embedded each month (1 million updates).
• Two layouts: one big index, or 100 tenants of 100,000 chunks each.

Pinecone Standard (pricing, as shown on October 15, 2026): $0.33/GB-month storage, $16 to $18 per million read units, $4 to $4.50 per million write units, egress at $0.10/GB after 100 GB a month included, and a $50 monthly minimum. I used the low end. Per Pinecone’s cost docs, a query costs 1 read unit per GB of namespace size, and an update costs 1 write unit per KB of the new plus old record, rounded up per request. I assume batches of 100 records. Storage: 72 GB × $0.33 = $23.76. Writes: 1M × 14.4 KB = 14.4M units × $4/M = $57.60. Reads in one namespace: 72 RU × 1M = 72M units, or $1,152. In 100 tenant namespaces: 0.72 RU × 1M = 0.72M units, or $11.52. Egress: 20 results with metadata is about 21 KB per query, so 21 GB at 1M queries (included) and 211 GB at 10M, or $11.12.

S3 Vectors, US East (S3 pricing): storage $0.06/GB-month, uploads $0.20/GB, $2.50 per million queries, plus a per-TB charge on the index data each query scans ($0.004/TB for the first 100,000 vectors, then $0.002/TB up to 10 million). I used binary units, as AWS’s examples do. Each PUT bills at least 128 KB, so I assume batches of at least 19 vectors (up to 500 are allowed per call). Storage: 67.06 GiB × $0.06 = $4.02. Churn uploads: 6.71 GiB × $0.20 = $1.34. A query on one 10M index scans 0.0655 TiB, costing $0.000132. That is $132.28 for 1M queries, plus $2.50 in fees. On 100,000-chunk tenant indexes, scan charges drop to $2.62.

pgvector on Supabase (compute pricing): a halfvec index of about 31 GB plus graph overhead. My estimate is that 4XL (64 GB, about $960/month) is the smallest size worth testing, and 8XL (128 GB, about $1,870) is the safe choice. Disk: 200 GB at $0.125/GB after 8 GB included, or $24. This excludes the plan fee and a failover replica, which I would budget at roughly double compute.

A single 72 GB Pinecone namespace costs more than the pgvector 4XL above about 784,000 queries a month. Split into 100 namespaces, the break-even moves to about 67 million, counting egress. Batching matters. Sent one record per request, Pinecone writes rise to $60 a month, and S3 churn uploads rise from $1.34 to $24.41. The one-time 10M load goes from $288 to $320 on Pinecone, and from $13.41 to $244.14 on S3 Vectors. These prices ignore latency, and S3 Vectors is built for infrequent queries. For the full budget, see what an AI agent costs to run per month.

Why vendor benchmarks mislead, and how to run your own

A vendor benchmark answers a question the vendor chose: QPS at a flattering recall level, its own system tuned and competitors on defaults, no filters or live writes, warm caches. Any one of those can flip a ranking.

Two public suites help if you know their limits. ANN-Benchmarks measures ANN search speed and recall. It includes adapters for databases such as pgvector, Qdrant, and Weaviate, but it does not test full production behavior like filters, live writes, or concurrency. It runs single-CPU on in-memory datasets of roughly 100 to 1,000 dimensions, with results as of April 2025, and its README says it is “no longer actively maintained.” VectorDBBench is closer to production. It tests filtered search, search during inserts, and concurrency. Zilliz, the company behind Milvus, sponsors it, and servers run on default settings. Treat it as a test kit to run yourself, not a verdict.

My one-week bake-off, using thresholds I set (they are not from a standard):

1. Day 1. Write 200 real queries with labeled relevant chunks, including filtered and identifier queries.
2. Day 2. Load a 10% sample of your corpus into two or three candidates. Compute exact nearest neighbors by brute force as ground truth.
3. Day 3. Measure recall@20 against exact search (target 0.95 or higher), p95 latency at twice your peak concurrency, and starved filtered queries as defined above (target zero).
4. Day 4. Re-embed 1% of the sample while running queries. Watch p95 and freshness.
5. Day 5. Rerun the cost table above with your own record size, query volume, and layout.

The AI agent evaluation guide covers labeled query sets. If retrieval passes and answers still miss, see agentic RAG vs traditional RAG and fine-tuning vs RAG.

Bottom Line

Pick from your constraints: vector count, filters, keyword needs, tenancy, latency, residency.

Do the byte math before you pick. RAM decides pgvector. Namespace size decides usage-based bills.

Run a one-week bake-off on your own queries. Ignore charts that hide recall, filters, and hardware.

FAQ

Can S3 Vectors be my only vector store for RAG?

For low-volume, vector-only retrieval, yes. Filterable metadata is capped at 2 KB per vector, and overwritten or deleted vectors can stay in billed index size for up to a day, which temporarily raises query costs.

Does a larger top_k cost more on Pinecone?

Not in read units. Pinecone’s cost docs say top_k, include_metadata, and include_values do not change a query’s read units. Only namespace size does. But bigger responses add egress, billed at $0.10/GB after 100 GB a month on Standard. Fetching records by ID costs 1 read unit per 10 records, rounded up.

Should each customer get its own Qdrant collection?

Usually not. Qdrant recommends one collection with a tenant field, and staying under roughly 1,000 dedicated shards per cluster. For BM25 queries, a tenant filter alone does not isolate term statistics. The idf search parameter (v1.19+) scopes them per tenant.

How fresh are writes in turbopuffer?

Queries are strongly consistent by default, and write latency is about 165 ms p50 for a 500 kB batch. Eventual consistency speeds up warm queries but can be up to about an hour stale.

Three Ways to Work With TMA

Need an agent built? We deploy production AI agents in your infrastructure. Working pilot. Real data. Measurable ROI. → Schedule Demo

Want to co-build a product? We’re not a dev agency. We’re co-builders. Shared cost. Shared upside. → Partner with Us

Want to join the Guild? Ship pilots, earn bounties, share profit. Community + equity + path to exit. → Become an AI Architect