Vector databases

Beginner

What problem does a vector database solve?

A vector database stores vectors and answers similarity questions such as "Which stored items are closest to this query vector?" The challenge is scale: comparing a query against every stored vector becomes too slow once you have hundreds of thousands or millions of records. Vector databases solve that by building indexes that avoid brute-force scans while still returning highly relevant neighbors.

• Each record usually contains an ID, a dense vector, optional metadata, and a pointer to source data.
• The query is converted into a vector using the same representation scheme as the stored records.
• The database returns the nearest neighbors according to a similarity function such as cosine, dot product, or Euclidean distance.
• Metadata filters narrow the candidate set, for example by tenant, language, product line, or time range.

Vector index vs. vector database

FAISS is a classic example of a vector index library. Systems such as Pinecone, Qdrant, Weaviate, Milvus, and pgvector-backed deployments add database capabilities around indexing so applications can manage vector data over time instead of treating search as a one-off algorithm.

Application record
    -> id
    -> vector
    -> metadata
    -> source pointer

Stored in vector database
    -> vector index for similarity search
    -> metadata store for filters
    -> storage/replication for persistence

Simple record structure

records = [
    {
        "id": "doc-001",
        "vector": [0.12, 0.55, -0.20],
        "metadata": {"tenant": "acme", "topic": "warranty", "year": 2025},
        "payload": {"title": "Warranty policy"}
    },
    {
        "id": "doc-002",
        "vector": [0.10, 0.51, -0.18],
        "metadata": {"tenant": "acme", "topic": "returns", "year": 2025},
        "payload": {"title": "Returns policy"}
    }
]

query_vector = [0.11, 0.52, -0.19]

# The database uses an index to find likely nearest neighbors quickly.

Intermediate

Similarity metrics

Nearest-neighbor search depends on how closeness is defined. The metric must match the assumptions of the embedding model and the index configuration.

Why approximate nearest neighbor search is necessary

Exact search checks every vector, which is feasible for small datasets but expensive for large collections. Approximate nearest neighbor (ANN) methods search only promising regions of the space, trading a small amount of recall for major latency and throughput gains.

The key operating knobs differ by index. For HNSW, search breadth affects recall and latency. For IVF, the number of coarse clusters and probes matters. For PQ, codebook size and compression ratio matter. Good teams benchmark these choices rather than assuming one default fits all datasets.

Filtering and data layout

Vector search almost never runs on vectors alone. Real applications need filters such as tenant separation, language, content type, freshness windows, or authorization tags. That means the database must coordinate vector search with structured filtering.

upsert_request = {
    "points": [
        {
            "id": "faq-17",
            "vector": [0.32, -0.14, 0.88],
            "metadata": {"tenant": "acme", "region": "us", "status": "published"}
        }
    ]
}

search_request = {
    "vector": [0.30, -0.10, 0.90],
    "top_k": 5,
    "filter": {"tenant": "acme", "status": "published"},
    "include_metadata": True
}

Write path
    application -> validate schema -> store vector + metadata -> update index

Read path
    query vector -> apply filters -> ANN candidate search -> score/rerank -> return top-k

Advanced

Production architecture concerns

A serious vector database is not just an index in memory. It must keep data durable, searchable after restarts, and responsive under changing traffic. That introduces classic distributed systems concerns on top of ANN search.

• Sharding: Partition data across nodes so the collection can grow beyond a single machine.
• Replication: Keep copies of data for fault tolerance and higher read throughput.
• Consistency: Decide how quickly writes must become visible across replicas.
• Freshness: Handle newly inserted vectors quickly even if the main index needs slower rebuild or compaction work.
• Multi-tenancy: Prevent one tenant's scale or hot traffic from degrading another tenant's queries.
• Access control: Restrict which tenants, users, or services can read specific vector collections.

What to measure

Benchmarking vector databases requires both search quality and systems metrics. Fast answers are not useful if the nearest neighbors are wrong, and accurate answers are not useful if p95 latency breaks your SLA.

Common failure modes

• Using the wrong metric: Cosine, dot product, and L2 are not interchangeable unless the embedding setup makes them equivalent.
• Ignoring filter design: Poor metadata schema can make filtered search much slower than unfiltered search.
• Assuming low average latency is enough: Tail latency usually matters more than the mean for user-facing systems.
• Over-compressing too early: Aggressive quantization or PQ can save memory while silently hurting recall.
• No ground-truth benchmark: Without exact-search comparisons on a sample set, ANN tuning becomes guesswork.

Selection checklist

If your workload needs:
    highest recall with enough RAM -> HNSW is often a strong default
    lower memory footprint at larger scale -> consider IVF/PQ variants
    strict tenant isolation -> prioritize namespaces, ACLs, and filter performance
    frequent writes -> verify upsert cost and freshness guarantees
    low-ops deployment -> managed/serverless offerings may matter more than raw ANN speed