← All problems
Problem 02 · Retrieval

Filter selectivity collapse.

The question

The same catalog search works fine for broad queries but returns near-empty or nonsensical results whenever a user applies three or more filters. Recall drops off a cliff below roughly 1% selectivity. Diagnose and fix.

The constraint. HNSW is a graph. Filtering restricts which nodes are admissible, but the graph's edges were built over the unfiltered corpus. At low selectivity the traversal walks through a neighbourhood where almost nothing qualifies, exhausts its candidate budget, and terminates early with whatever it happened to touch. This is a property of the index, not a bug in your code.

Three regimes, three answers.

SelectivityStrategy
> 10%Filtered traversal is fine. Push the predicate into the index.
1-10%Filtered traversal with an inflated ef_search, or IVF with partition pruning aligned to the filter.
< 1%Stop using ANN. Fetch the filtered set exactly and brute-force the distances.

The last row is the answer people miss. Below 1% of 500M is 5M rows, but a specific three-predicate filter usually resolves to hundreds or thousands of rows. Exact scan over 2,000 vectors is sub-millisecond. ANN is an optimisation for large candidate sets; when the candidate set is small, it is pure loss.

Architecture. Estimate selectivity before choosing a path. Postgres already keeps the statistics; a cardinality estimate from the filter predicate is enough to pick a regime. Route accordingly:

  1. 01Cheap cardinality estimate on the structured predicate.
  2. 02High selectivity → filtered ANN.
  3. 03Low selectivity → materialize IDs from the relational store, fetch vectors by ID, exact distance, done.
  4. 04Partition the index along the highest-cardinality filter you actually use (tenant, category, region) so pruning happens at partition level rather than inside traversal.

Stack. Qdrant exposes payload indexes and lets you tune ef per query. Vespa expresses this natively as ranking phases with a filter-first query plan. Elasticsearch and OpenSearch expose num_candidates for the same knob. pgvector with a partial index per tenant handles the small-partition case well and keeps everything in one system.

Where answers fail. Treating it as a tuning problem. No value of ef_search rescues a 0.1% filter; the fix is to not use the graph.