Retrieval

Find what belongs in the window

A model has no filing cabinet. Retrieval is the clerk who fetches the right folder before anyone asks the question.

A language model only knows what is put in front of it. It cannot go and look something up. So before you ask it anything, something has to walk into your company's documents, find the handful of paragraphs that matter, and paste them into the question. That fetching job is retrieval. Get it wrong and the model answers confidently from nothing — which is what people mean when they say it made something up.

46 entries · 6 categories · 4 equations

How it actually works

no jargon

PDFs, wikis, tickets and spreadsheets get turned into clean text. It is the boring step that quietly decides everything after it — a table mangled here becomes a wrong answer later.

The arithmetic

Reciprocal rank fusion

\mathrm{RRF}(d) \;=\; \sum_{r \in R} \frac{1}{k + r(d)}, \qquad k = 60

How Elasticsearch and OpenSearch fuse BM25 with dense results. It uses ranks, not scores, so you never have to normalise incomparable similarity scales. k = 60 comes from Cormack et al. 2009 and damps top-rank dominance — a document ranked 1st lexically and 50th semantically still surfaces.

Late interaction — MaxSim

S_{q,d} \;=\; \sum_{i \in |q|} \max_{j \in |d|} \; E_{q_i} \cdot E_{d_j}^{\top}

ColBERT scores every query token against its best-matching document token instead of collapsing both to one vector. Recall improves; the index stores a vector per token, so it runs one to two orders of magnitude larger. That storage bill is the whole trade.

HNSW resident memory

M \;\approx\; N\,(4d + 8m)

N vectors of d float32 dimensions plus roughly m graph links per node. Ten million 1536-dim vectors at m = 16 need ~63 GB of RAM. That figure — not query latency — is what pushes teams to quantization or to object-storage-native engines.

Recall / latency knob

\text{recall} \nearrow \text{ saturating},\quad \text{latency} \sim \mathcal{O}(\mathrm{efSearch}\cdot \log N)

Recall saturates in efSearch while latency keeps climbing roughly linearly. Measure the knee on your own data; the defaults shipped by every vector database are tuned for benchmark recall, not your p99.

Vector databases

Approximate nearest-neighbour storage and filtering.

FAISS

ANN index library rather than a database; baseline for offline search

libraryoffline

MITopen sourcewidely deployedself-host

Milvus / Zilliz

Distributed billion-scale vector database; an LF AI & Data project

distributedbillion-scale

Apache-2.0open corewidely deployedboth

MongoDB Atlas Vector Search

Vector index inside MongoDB with native embedding and rerank APIs

manageddocument-db

Commercial / SSPL corecommercialwidely deployedsaas

pgvector

Postgres extension: HNSW/IVFFlat vector indexes in your existing database

postgresself-hostdefault-choice

PostgreSQLopen sourcewidely deployedself-host

Pinecone

Serverless managed vector database with hybrid search and namespaces

managed

Commercialcommercialwidely deployedsaas

Qdrant

Rust vector database with payload filtering, quantization and on-disk HNSW

rustfiltering

Apache-2.0open corewidely deployedboth

Vespa

Search and ranking engine with tensor compute and ML ranking phases

rankinglarge-scale

Apache-2.0open corewidely deployedboth

Weaviate

Native BM25 and dense hybrid search, multi-tenancy, module system

hybridmulti-tenant

BSD-3open corewidely deployedboth

Chroma

Embedded single-node vector store with a Rust core

embeddedprototyping

Apache-2.0open coreproduction viableboth

LanceDB

Serverless vector database on the Lance columnar format, reads from object storage

object-storageserverless

Apache-2.0open coreproduction viableboth

pgvectorscale

StreamingDiskANN index layered on pgvector for larger-than-memory workloads

postgresdiskann

PostgreSQLopen sourceproduction viableself-host

Turbopuffer

Object-storage-native search with SPFresh index, BM25 and sparse vectors

object-storagecost-efficient

Commercialcommercialproduction viablesaas

VectorChord

Postgres extension with disk-friendly IVF/RaBitQ; successor to pgvecto.rs

AGPL — check policy before embedding.

postgresagpl

AGPL-3.0 + commercialopen coreproduction viableself-host

Search & hybrid retrieval

Lexical and dense retrieval fused.

Elasticsearch

BM25 plus dense and sparse retrieval, RRF hybrid, retrievers API

Added AGPL-3.0 in 2024, restoring OSI-approved status.

incumbenthybrid

AGPL-3.0 / ELv2 / SSPLsource availablewidely deployedboth

OpenSearch

Apache-2.0 fork with k-NN, RRF hybrid and neural sparse retrieval

incumbentlinux-foundation

Apache-2.0open sourcewidely deployedboth

Tantivy

Rust full-text search library embedded in many higher-level tools

libraryrust

MITopen sourcewidely deployedself-host

Meilisearch

Lightweight full-text and hybrid search for in-application search

app-search

MITopen coreproduction viableboth

ParadeDB

Postgres extension adding Tantivy-backed BM25 for in-database hybrid search

AGPL — check policy before embedding.

postgresagpl

AGPL-3.0 + commercialopen coreproduction viableself-host

Typesense

Typo-tolerant keyword search with vector fields and a simple ops model

app-search

GPL-3.0open coreproduction viableboth

Embeddings & rerankers

Turning text into vectors, then reordering results.

BGE (BAAI)

Open-weight embeddings; bge-reranker-v2-m3 is the common self-host default

open-weightsdefault-choice

MIT / Apache-2.0open sourcewidely deployedself-host

Cohere Embed v4 / Rerank 4

Multimodal embeddings with Matryoshka dims, plus a cross-encoder rerank API

managedmultimodal

Commercialcommercialwidely deployedsaas

ColBERT / PLAID

Late-interaction retrieval with token-level MaxSim; higher index cost

late-interactionresearch-derived

Apache-2.0open sourceproduction viableself-host

FlashRank

Small CPU-only reranker for latency-constrained pipelines

cpulow-latency

Apache-2.0open sourceproduction viableself-host

Jina AI

Open-weight embeddings and multimodal rerankers that score page images

open-weightsmultimodal

Mixed OSS + APIopen coreproduction viableboth

Mixedbread

Open-weight embedding and reranker models plus a hosted API

open-weights

Apache-2.0open coreproduction viableboth

Qwen3-Embedding / Reranker

Open-weight embedding and reranker models topping MTEB in 2026

open-weightssota

Apache-2.0open sourceproduction viableself-host

Voyage AI

Domain-tuned embeddings and rerankers for legal, code and finance

Acquired by MongoDB, February 2025.

manageddomain-tunedacquired

Commercialcommercialproduction viablesaas

ZeroEntropy

zerank reranker models at the top of several 2026 leaderboards

open-weightsreranker

OSS weights + APIopen coreearlyboth

Parsing & ingestion

Getting messy documents into clean chunks.

LlamaParse

VLM-based PDF parsing to Markdown; best-in-class on messy documents

pdfvlm

Commercialcommercialwidely deployedsaas

Unstructured

Thirty-plus formats to normalized elements, with connectors and VPC deployment

connectorsvpc

Apache-2.0 coreopen corewidely deployedboth

Chonkie

Chunking library covering token, semantic, recursive and late chunking

chunkinglibrary

MITopen sourceproduction viableself-host

Docling

IBM Research layout and table pipeline; local PDF-to-Markdown, no per-page cost

pdflocalno-per-page-cost

MITopen sourceproduction viableself-host

Firecrawl

Crawls sites to clean Markdown or JSON and handles JS rendering

AGPL — check policy before embedding.

crawlingagpl

AGPL-3.0 coreopen coreproduction viableboth

Marker

PDF and EPUB to Markdown, GPU-accelerated and batch-oriented

pdfgpu

GPL-3.0 + commercialsource availableproduction viableself-host

Reducto

Agentic OCR-correction parsing with on-prem, SOC 2 Type II and HIPAA

complianceon-prem

Commercialcommercialproduction viableboth

Chunkr

Parsing and semantic chunking API, self-hostable

AGPL — check policy before embedding.

chunkingagpl

AGPL-3.0open coreearlyboth

Managed retrieval

Retrieval as a hosted service.

Azure AI Search

Hybrid BM25, vector and semantic reranking with integrated vectorization

hyperscalerazure

Commercialcommercialwidely deployedsaas

Bedrock Knowledge Bases

Managed ingestion and retrieval over OpenSearch, Aurora or Neptune

hyperscaleraws

Commercialcommercialwidely deployedsaas

Vertex AI Search

Managed retrieval and grounding with connectors and a ranking API

hyperscalergcp

Commercialcommercialwidely deployedsaas

Vectara

Managed RAG with HHEM hallucination scoring; enterprise-only since 2026

Self-service tier discontinued in 2026.

managed-ragenterprise-only

Commercialcommercialproduction viablesaas

Ragie

Managed ingestion and retrieval API with connectors and citations

managed-rag

Commercialcommercialearlysaas

Graph retrieval

Entity and relationship structure over a corpus.

Graphiti

Temporal knowledge graph with edge validity intervals and hybrid graph-vector search

graphtemporal

Apache-2.0open sourceproduction viableboth

Microsoft GraphRAG

Entity extraction, community detection and hierarchical summaries

Indexing cost is substantial at corpus scale.

graphglobal-queries

MITopen sourceproduction viableself-host

neo4j-graphrag-python

Official Neo4j retrievers and knowledge-graph construction pipeline

graphneo4j

Apache-2.0open sourceproduction viableself-host

Cognee

Pipeline turning documents into a queryable memory graph

graphmemory

Apache-2.0open sourceearlyself-host

LightRAG

Flat dual-level graph index, far cheaper to build than GraphRAG

graphcost-efficient

MITopen sourceearlyself-host

Field notes

What enterprises actually run is Elasticsearch/OpenSearch, pgvector, or a hyperscaler retrieval service. Most 'which vector DB' debates resolve to 'the one your database or cloud vendor already ships'.
Postgres covers workloads into the tens of millions of vectors. Move to a dedicated engine past that, or when filter-heavy QPS demands it.
Adding a cross-encoder rerank stage usually beats swapping vector databases. Open weights are competitive with commercial APIs, so lock-in here is low.
Object-storage-native architectures have cut cost-per-vector by roughly an order of magnitude against in-memory HNSW.
Licensing moved toward openness, unusually: Elasticsearch added AGPL-3.0 in 2024. But VectorChord, ParadeDB, Chunkr and Firecrawl are AGPL — check policy before embedding.
Standalone vector DBs are consolidating. Voyage AI went to MongoDB, Quickwit to Datadog. Assume further absorption.
Superseded: pgvecto.rs, replaced by VectorChord.