--- source_url: "https://fastcrw.com/blog/best-vector-databases?utm_source=openai" title: "Best Vector Databases in 2026: A Complete Comparison | fastCRW" mirrored_at: 2026-08-13T01:10:34.705Z host: fastcrw.com cited_in_42a: true mirror_canonical: "https://index.42a.ai/fastcrw.com/blog/best-vector-databases__q__utm_source_openai" --- > **Original source:** https://fastcrw.com/blog/best-vector-databases?utm_source=openai _By the fastCRW team · Vector-DB facts compiled from public vendor docs and benchmarks, verified 2026-05-29 · Verify vendor numbers independently before you commit._ **Disclosure:** We build fastCRW, an open-core scraper/crawler. We do not sell a vector database, so we have no horse in the pgvector-vs-Pinecone race — which is exactly why we can give you a straight comparison and then be honest about the one part we do care about: the ingestion layer that feeds whichever store you pick. ## The best vector databases in 2026, and the one variable they all share Choosing the best vector database for a RAG or semantic-search app comes down to four axes: scale (how many vectors), hosting (managed vs self-hosted), cost (at your real traffic), and filtering (hybrid search, metadata, multi-tenancy). The six names you will keep seeing — **pgvector, Pinecone, Qdrant, Weaviate, Milvus, and Chroma** — cover the overwhelming majority of production deployments, and we compare them head-to-head below. But there is a variable that sits upstream of every one of them and that no vector-database benchmark measures: the quality of what you embed. A vector store can only retrieve what you put in it. Garbage chunks — boilerplate navigation, duplicated headers, broken markdown, the same paragraph indexed three times — produce garbage retrieval no matter how low your p99 latency is. That ingestion layer is where fastCRW lives, and we will get to it. First, the databases. ## What a vector database does in a RAG stack A vector database stores embeddings — arrays of floats that encode the meaning of a chunk of text — and finds the nearest ones to a query embedding using approximate nearest-neighbour (ANN) search. In a retrieval-augmented-generation pipeline it sits between your ingestion job and your LLM: you embed your documents once, then at query time you embed the user's question, pull the top-k nearest chunks, and stuff them into the prompt as context. The core trade-off every engine makes is **recall vs speed**. Exact nearest-neighbour search is too slow at scale, so all of these systems use approximate indexes (most commonly HNSW, a layered proximity graph) that trade a sliver of accuracy for large speed gains. A benchmark number is meaningless without its recall figure attached: "10 ms at 90% recall" and "40 ms at 99% recall" are solving different problems. When you read the vendor latency claims below, mentally pin each one to its recall level. If you are still assembling the surrounding pieces — chunker, embedder, retriever, reranker — our [best RAG tools](https://fastcrw.com/blog/best-rag-tools) roundup covers the full stack, and [building a RAG pipeline with CRW](https://fastcrw.com/blog/rag-pipeline-with-crw) walks the ingestion half end to end. ## Vector database comparison table The numbers below are vendor-published or third-party benchmark figures (sources at the end); treat them as directional and re-run on your own workload before quoting them internally. Database Type Best scale Hosting Filtering / hybrid Cost shape pgvector + pgvectorscale PostgreSQL extension < ~100M vectors Self-host or any managed Postgres SQL `WHERE` + vector; no native BM25 Your existing Postgres bill Qdrant OSS (Rust) + managed < ~50M vectors Self-host or Qdrant Cloud Rich JSON filters, native hybrid Free 1GB tier; paid from low monthly Milvus / Zilliz OSS + managed Billions Self-host (k8s) or Zilliz Cloud Filtering + hybrid; needs ops Infra-only self-host; managed adds margin Weaviate OSS + managed < ~100M vectors Self-host or Weaviate Cloud Best-in-class hybrid (vector + BM25 + filters) OSS free; cloud is dimension-based Pinecone Managed only 10M–billions, zero ops Serverless cloud Metadata filtering, namespaces Usage-based (storage + read/write units) Chroma OSS (embedded) Prototypes, < ~10M Embedded / self-host / cloud Metadata + full-text, simple Free OSS; cloud credits ## Open-source, self-hosted options (pgvector, Qdrant, Milvus, Chroma) If you want a hard cost ceiling and data that never leaves your infrastructure, the self-hostable engines are where you start. They map cleanly to four different situations. ### pgvector — you already run PostgreSQL If your app already has a Postgres database, `pgvector` plus the `pgvectorscale` extension is the lowest-friction option on this list: vectors live in the same table as your relational data, you query both in one transaction, and you add zero new infrastructure. The performance reputation has shifted dramatically — Timescale's published benchmark reports pgvectorscale hitting 471 QPS at 99% recall on 50M vectors, an order of magnitude above an earlier Qdrant measurement on the same dataset (Tigerdata benchmark, 2025). The honest ceiling: beyond ~100M vectors the relational storage model starts to lose to purpose-built engines, and ORM support (e.g. Prisma partitioning) is still patchy. Under 50M vectors, for most teams, it is the pragmatic default. ### Qdrant — Rust, best free tier, great filtering Qdrant is a Rust-based vector engine with a generous 1GB-forever free tier and the cleanest metadata-filtering story of the budget options — rich JSON filters that the query planner folds into the search efficiently. Its compact footprint suits edge and small-instance deployments. The published trade-off is throughput at scale: third-party benchmarks show it falling well behind disk-based approaches past ~10M vectors. For a sub-50M-vector RAG app where filtering by document type, tenant, or date matters, it is an excellent self-hosted pick. ### Milvus — billions of vectors, if you have the ops budget Milvus is the heavyweight: a distributed, cloud-native engine that separates storage and compute and is proven at billions of vectors. That power comes with a Kubernetes-shaped operational burden — you tune HNSW parameters, debug distributed failures, and staff for it. The pay-off is cost at scale: self-hosting on your own infra is dramatically cheaper than managed alternatives once you are in the hundreds-of-millions range, and Zilliz Cloud exists if you want the architecture without running it. Pick Milvus when vectors are your primary, billion-scale workload and you have data engineers. ### Chroma — prototyping and MVPs Chroma optimises for one thing: getting a RAG prototype working this afternoon. The Python API feels like NumPy, it runs embedded in your process with no network hop, and it is Apache-2.0 free. The 2025 Rust rewrite made it materially faster, but it is still explicitly not built for production scale — teams routinely start on Chroma and migrate to Qdrant, Pinecone, or Milvus when they cross into the millions. Treat it as the dev-and-demo store, not the one you scale on. ## Managed options (Pinecone, Weaviate Cloud) If you would rather trade dollars for never paging at 3am over a distributed-systems failure, the managed engines remove the operational surface entirely. ### Pinecone — zero-ops at any scale Pinecone is the reference managed vector database: create an index, upsert vectors, query, and let it auto-scale from hundreds to tens of thousands of QPS without you touching configuration. Its published p99 latency is in the single-digit milliseconds, and it is proven at billions of vectors. The two trade-offs are well known and real: usage-based pricing (storage plus read/write units) that climbs steeply for high-throughput workloads, and proprietary lock-in — migrating off means rebuilding the integration. You choose Pinecone when time-to-market and zero ops beat cost optimisation. ### Weaviate Cloud — hybrid search as a first-class citizen Weaviate's differentiator is hybrid search: vector similarity, BM25 keyword matching, and metadata filters fused in a single query, built into the core rather than bolted on. For RAG over messy real-world corpora — where a pure-vector query alone misses the exact-term matches users expect — this is genuinely valuable, and the documentation is among the best in the category. The honest notes: a short trial window on the cloud tier relative to Qdrant's permanent free tier, a GraphQL-first API some teams dislike, and rising resource needs above ~100M vectors. Below that, for hybrid-search use cases, it is hard to beat. ## Choosing by scale and use case Most engines deliver sub-100ms queries under 10M vectors; the differences only start to bite at 50M and become decisive past 100M. Unless you have a concrete scaling plan inside six months, solve for the scale you actually have today. A rough decision path: - **Already on PostgreSQL, under 50M vectors:** start with pgvector + pgvectorscale. One fewer system to run. - **New project, under 10M, budget-tight:** Qdrant (best free tier) for real work, Chroma for the prototype. - **RAG that needs keyword + vector + filters together:** Weaviate for the native hybrid search. - **Want managed and zero ops, 10M–billions:** Pinecone serverless. - **Billions of vectors, you have data engineers and want the lowest infra cost:** self-hosted Milvus (or Zilliz Cloud for managed Milvus). - **Data cannot leave your network (compliance):** any of the self-hostable engines — pgvector, Qdrant, Milvus, Chroma — ruling out the managed-only options entirely. Notice that last row is binary: a residency or compliance requirement can disqualify managed-only Pinecone before any benchmark matters. The same logic applies one layer up, to how the data gets _into_ the database in the first place. ## Feeding clean web data into any vector database Here is the part vector-DB comparisons skip. Every benchmark above assumes the embeddings are good. But in a web-sourced [RAG pipeline](https://fastcrw.com/use-cases/ai-chat-rag) the embeddings are only as good as the markdown you fed the embedding model — and raw HTML is hostile to that. Nav bars, cookie banners, footers, and ad markup all embed into noise vectors that pollute retrieval; the same article reachable at three URLs gets indexed three times and crowds out everything else in your top-k. The shape of the ingestion pipeline is identical no matter which database you picked above, which is the whole point: **scrape or crawl → clean markdown → chunk → embed → upsert.** Swap the last step and the first four are unchanged. fastCRW is the open-core engine for those first steps — a single static Rust binary (~8 MB, one container, versus a multi-container browser stack) that turns pages into LLM-ready markdown, with crawl-level dedup so the same content does not land in your index twice. See [scrape-to-RAG with LlamaIndex](https://fastcrw.com/blog/scrape-to-rag-pipeline-llamaindex) for a worked end-to-end build. Because fastCRW is Firecrawl-compatible, you point the standard SDK at your own engine with a base-URL swap, then hand the markdown to whatever embedder and store you chose: ``` from firecrawl import FirecrawlApp # Self-hosted fastCRW engine — same SDK, your URL app = FirecrawlApp(api_url="http://localhost:3002", api_key="local") doc = app.scrape_url("https://example.com/docs/guide", formats=["markdown"]) clean_markdown = doc["markdown"] # ready to chunk + embed into ANY store # ...your chunker -> embedder -> pgvector / Qdrant / Pinecone upsert here ``` On extraction quality we cite real numbers, not marketing rounding. On Firecrawl's own public 819-labeled-URL dataset, fastCRW scored the highest truth-recall of the three engines tested — **63.74%** (522 of 819) versus 59.95% and 56.04% for the other two — at a median scrape latency of **1914 ms** p50 (`diagnose_3way.py`, 2026-05-08). Truth-recall is "did we extract the labeled ground-truth content," which is exactly the variable that decides whether a retrieved chunk is useful or empty. The tail holds up too: in fast mode that same run's **p90 is 4348 ms — the lowest tail of the three** (Crawl4AI 4754 ms, Firecrawl 6937 ms), because the stealth fallback that recovers hard pages is the same mechanism that keeps the tail competitive. Two more ingestion realities. First, **privacy**: self-hosted, the scraped content and target URLs never leave your network — the same residency requirement that pushes you toward a self-hostable vector DB pushes you toward a self-hostable scraper. Our note on [local-first scraping and data privacy](https://fastcrw.com/blog/local-first-scraping-data-privacy) covers why. Second, **cost ceiling**: self-hosting the AGPL-3.0 engine is $0 per 1,000 scrapes beyond your own server bill; on the managed cloud, [fastCRW pricing](https://fastcrw.com/pricing) runs Hobby $13 / Standard $69 / Growth $279 / Scale $549 per month, with a Free tier of 500 one-time credits — current numbers live at [/pricing](https://fastcrw.com/pricing). ## What fastCRW is not fastCRW is the ingestion layer, not a vector database — it does not store or search embeddings, and you still bring your own embedder and store from the list above. On the scraping side, it renders and returns a CDP screenshot alongside markdown and JSON, runs structured JSON extraction across up to 50 URLs in one `/v1/extract` call, and ships a Research API (`/v2/search/research/papers`) for arXiv/OpenAlex/Semantic Scholar fan-out. LLM-based structured extraction runs on fastCRW's managed LLM and requires a paid plan. ## Sources - fastCRW extraction benchmark: `diagnose_3way.py` on Firecrawl's public 1,000-URL `scrape-content-dataset-v1`, 819 labeled URLs, single run 2026-05-08 — truth-recall 63.74% (highest), p50 1914 ms (fastest), p90 4348 ms in fast mode (lowest). fastCRW repo: [github.com/us/crw](https://github.com/us/crw) · benchmarks: [/benchmarks](https://fastcrw.com/benchmarks) · pricing: [/pricing](https://fastcrw.com/pricing) - Vector-DB figures are vendor-published or third-party (Tigerdata/Timescale pgvectorscale benchmark 2025; Zilliz VectorDBBench; vendor docs for Pinecone, Qdrant, Weaviate, Milvus, Chroma), verified 2026-05-29. Re-run on your own workload before quoting. Related: [Best RAG tools](https://fastcrw.com/blog/best-rag-tools) · [RAG pipeline with CRW](https://fastcrw.com/blog/rag-pipeline-with-crw) · [Scrape-to-RAG with LlamaIndex](https://fastcrw.com/blog/scrape-to-rag-pipeline-llamaindex) · [Local-first scraping & data privacy](https://fastcrw.com/blog/local-first-scraping-data-privacy)