Filter selectivity collapse
The same catalog search works fine for broad queries but returns near-empty or nonsensical results whenever a user applies three or more filters. Recall drops off a cliff below roughly 1% selectivity. Diagnose and fix.
The constraint. HNSW is a graph. Filtering restricts which nodes are admissible, but the graph's edges were built over the unfiltered corpus. At low selectivity the traversal walks through a neighbourhood where almost nothing qualifies, exhausts its candidate budget, and terminates early with whatever it happened to touch. This is a property of the index, not a bug in your code.
Three regimes, three answers.
| Selectivity | Strategy |
|---|---|
| > 10% | Filtered traversal is fine. Push the predicate into the index. |
| 1-10% | Filtered traversal with an inflated ef_search, or IVF with partition pruning aligned to the filter. |
| < 1% | Stop using ANN. Fetch the filtered set exactly and brute-force the distances. |
The last row is the answer people miss. Below 1% of 500M is 5M rows — but a specific three-predicate filter usually resolves to hundreds or thousands of rows. Exact scan over 2,000 vectors is sub-millisecond. ANN is an optimisation for large candidate sets; when the candidate set is small, it is pure loss.
Architecture. Estimate selectivity before choosing a path. Postgres already keeps the statistics; a cardinality estimate from the filter predicate is enough to pick a regime. Route accordingly:
- 01Cheap cardinality estimate on the structured predicate.
- 02High selectivity → filtered ANN.
- 03Low selectivity → materialize IDs from the relational store, fetch vectors by ID, exact distance, done.
- 04Partition the index along the highest-cardinality filter you actually use — tenant, category, region — so pruning happens at partition level rather than inside traversal.
Stack. Qdrant exposes payload indexes and lets you tune ef per query. Vespa expresses this natively as ranking phases with a filter-first query plan. Elasticsearch and OpenSearch expose num_candidates for the same knob. pgvector with a partial index per tenant handles the small-partition case well and keeps everything in one system.
Where answers fail. Treating it as a tuning problem. No value of ef_search rescues a 0.1% filter; the fix is to not use the graph.