What Is a Vector Database? (and When You Actually Need One)

You're building an AI feature. The model writes confident prose, but it can't see your private docs, your current inventory, or anything that happened after its training cutoff. So it makes things up. Every tutorial you find says the same thing: put your data in a vector database and feed the relevant bits back into the prompt. Fair enough. But that advice skips the question you actually have once you start pricing it out — do you need a dedicated vector database, or is the Postgres you already run enough?

This covers both. First what a vector database is and how it works, in plain terms. Then the part most articles skip: when you genuinely need a separate one, and when reaching for it is just extra infrastructure you'll regret.

What a vector database actually is

A relational database is built to answer "does this exact record exist?" — find the user with this ID, the orders placed last Tuesday, the rows where status equals 'shipped'. A vector database is built to answer a different question: "what is most similar to this?" That shift is the whole point. Keep it in mind and most of the design decisions downstream make sense.

The thing it stores is a vector embedding — a list of numbers, often hundreds or thousands of them, that represents a piece of unstructured content: a paragraph, an image, an audio clip. You get these vectors from a machine learning model. Send text to an embedding model (OpenAI's or Cohere's hosted ones, or an open model like BERT) and it returns a vector. Send an image through a model like CLIP and you get a vector in a comparable space. The useful property is geometric: content that means similar things produces vectors that sit close together. "How do I reset my password" and "I forgot my login" land near each other even though they share almost no words.

"Close together" needs a definition, and that's where distance metrics come in. A search ranks candidates by how near they are to your query vector, using cosine similarity (the angle between vectors), dot/inner product, or Euclidean distance. Which one you use depends on the embedding model — most text models are tuned for cosine. The database's job is to take a query vector and return the nearest neighbors fast.

That's the definition. A vector database stores embeddings, indexes them, and retrieves the closest matches to a query. Everything else is about making that retrieval fast at scale.

How it works under the hood

The naive approach is to compare your query against every stored vector and sort by distance. That's exact, and it's fine for a few thousand vectors. It falls apart at a few million, and it's hopeless at a billion — you'd be doing a billion distance calculations per query.

So vector databases don't do exact search. They use Approximate Nearest Neighbor (ANN) search, which trades a sliver of accuracy for an enormous speedup. "Approximate" means you might occasionally miss the true closest match in favor of the second-closest — recall in the high-90% range is typical and, for search and recommendations, completely acceptable. Nobody notices the 50th-best result being in 51st place.

The speedup comes from an index. Two families show up everywhere, and it's worth recognizing the names:

• HNSW (Hierarchical Navigable Small World) builds a layered graph you navigate from coarse to fine, hopping toward the query region instead of scanning everything. High accuracy, fast queries, and the de facto default in 2026. The tradeoff is memory — the graph lives in RAM.
• IVF (Inverted File Index) partitions vectors into clusters and, at query time, only searches the few clusters nearest your query. It scales well to very large datasets and is lighter on memory than HNSW, at some cost to recall.

There's a newer option worth knowing: DiskANN, a disk-based index for datasets too large to hold in RAM. It's now part of the pgvector ecosystem (via the pgvectorscale extension), which matters if your corpus outgrows memory but you don't want to shard.

Two more features are table stakes in 2026, not extras. Metadata filtering lets you say "find similar documents, but only ones belonging to this tenant" — combining a similarity search with a structured constraint. Hybrid search blends vector similarity with old-fashioned keyword matching, which catches the cases where exact terms matter (product codes, names, error strings) and pure semantic search drifts.

What you'd actually use it for

The headline use case is RAG — Retrieval-Augmented Generation. Instead of fine-tuning a model on your data (expensive, slow to update, easy to get wrong), you store your documents as embeddings and, at query time, retrieve the handful most relevant to the user's question and paste them into the prompt. The model answers from real source material it can see right now. That cuts hallucinations and means updating your knowledge base is just an upsert, not a retraining run.

Beyond RAG, the same machinery powers semantic search (find by meaning, so "laptop won't charge" matches a doc titled "power adapter troubleshooting"), recommendations (the "more like this" on products, articles, or media), and a long tail of similarity jobs — deduplication, anomaly detection, reverse image search, audio matching.

One honest note on RAG in 2026: the naive 2023 recipe — chunk your docs, dump them in a vector store, retrieve top-k, done — is widely considered outdated. Modern systems layer in hybrid search, a reranking step to reorder retrieved candidates, and sometimes long-context models that swallow more at once. The vector store is still the retrieval backbone. It's just no longer the whole system.

When you actually need a dedicated one

Here's the part the definition articles skip. Treat this as a decision, not a shopping trip.

You probably do NOT need a dedicated vector database if you already run PostgreSQL and your corpus is in the low millions of vectors. The pgvector extension handles this comfortably — with an HNSW index it delivers sub-20ms queries at around a million vectors with over 95% recall, and it's widely reported to stay comfortable up to roughly 50 million vectors. (Add pgvectorscale and benchmarks push further — one cites 471 QPS at 99% recall on 50M vectors. Vendor numbers, so read them as directional.) For documentation search, support-ticket classification, an internal knowledge base, or typical RAG, this is the default that fits most teams.

The real argument for staying in Postgres isn't the benchmark — it's operational. One system to back up. One place where your vectors and your relational data live, so a similarity search can join against your actual tables with transactional consistency. No extra network hop on every query, no second datastore to monitor, secure, and keep in sync. That's a lot of weight on the "don't add infrastructure" side of the scale.

You DO need (or clearly benefit from) a dedicated vector database if you hit one of these walls:

• Scale past what Postgres wants to hold — hundreds of millions to billions of vectors. Engines built only for this shard and serve at that size without a fight.
• Real-time indexing. If vectors must be searchable the instant they're ingested, pgvector's index builds can lag. Dedicated engines are designed for immediate availability.
• Single-digit-millisecond latency under heavy concurrency. In-memory specialized engines hold tight latency where a general-purpose database under load starts to wobble.
• Zero-ops managed infrastructure, or advanced built-in hybrid search and multi-tenancy at scale — the kind of thing Pinecone, Weaviate, Qdrant, and Milvus package up so you don't build it yourself.

The heuristic to carry out of here: start with the database you already have. Reach for a dedicated vector database when scale, latency-under-load, real-time indexing, or managed-ops needs force the issue — not before. Adding a specialized datastore on day one because a tutorial said so is how you end up maintaining two systems to serve ten thousand vectors.

The landscape, briefly

Positioning as of mid-2026, qualitative on purpose — latency and QPS rankings swing with workload, hardware, and whoever published the benchmark.

Native hybrid search (keyword + vector in one query) is supported in 2026 across Weaviate, Milvus 2.5+, Qdrant 1.9+, LanceDB, and Pinecone — it's no longer a differentiator, just an expectation.

Since cost is usually what tips the build-vs-buy call, the managed end is worth concrete numbers. Pinecone's published serverless pricing:

One correction worth making because it's been copied around: some write-ups list Pinecone storage at "$3.60/GB/month." The official pricing page says $0.33/GB/mo. Off by more than 10x — check the source before you put it in a budget.

Where this leaves you

Keep the mental model — a vector database answers "what's most similar?" — and keep the heuristic: start with what you already run, and let a real constraint, not a tutorial, push you toward dedicated infrastructure. The more interesting question in 2026 isn't "vector DB or not" anymore. It's how retrieval fits into the larger system around it: hybrid search, reranking, and agents that decide for themselves what to look up. The vector store is the foundation that makes all of that possible. It's rarely the hard part.

References

• Pinecone official pricing
• pgvector 0.8.2 released (PostgreSQL.org)
• pgvector 0.8.0 released (PostgreSQL.org)
• pgvector CHANGELOG (GitHub)
• You probably don't need a vector database (Encore)
• pgvector vs dedicated vector DB (zenvanriel)
• Vector database: how it works (Cloudian)
• Best vector databases 2026 (DataCamp)
• Vector databases explained in 3 levels (MachineLearningMastery)
• What is RAG in 2026 (Atlan)