Indexing → Filtered search (pre-filter vs post-filter)

Filtered Vector Search Explained: Pre-Filter vs Post-Filter

What it is

Three ways to combine "nearest vectors" with "must satisfy filter":

StrategyHowFails when
Post-filterANN top-K (say 100) → drop those failing the filter → return survivorsFilter is selective (tenant = 1% of corpus): most of the 100 are dropped; you get 0–3 results, or none
Pre-filter (brute force)Evaluate the filter first → exact search over the matching subsetSubset is large (filter matches 50% of 50M): a brute-force scan is slow
Filter-aware ANN (integrated)Graph traversal skips non-matching nodes, or partitions by filter value, or per-filter sub-indexesExtremely selective filters can strand the graph (disconnected matching nodes); needs the store's implementation to handle it (Qdrant, Weaviate, Milvus, Vespa, Pinecone, pgvector with recent versions do variations)

Good stores switch strategy by estimated selectivity: brute-force over small filtered sets, filter-aware graph search otherwise. The common production filters — tenant_id, is_current, acl_groups ∩ caller_groups, doc_type, date ranges — are often very selective per query (one tenant of thousands), which is exactly where naive post-filtering breaks.

How to implement

  1. Use a store with native filtered ANN (vector store choice) and create payload/metadata indexes on every filter field (keyword for IDs/enums, numeric/int for dates as epoch).
  2. Order filters by selectivity in your query builder (tenant first) — some engines benefit; all benefit from clarity.
  3. Set k and ef for filtered queries: with filters, raise ef_search (or nprobe) and request more candidates than you need; measure recall vs a filtered flat search (index types: Flat, HNSW, IVF-PQ).
  4. Partition when selectivity is extreme and stable: per-tenant collections/namespaces or the store's tenant sharding, so the "filter" becomes "choose the index".
  5. Never post-filter security fields in application code as the only mechanism — it both loses results and risks bugs; if you must post-filter (legacy store), over-fetch heavily (K = 10× k) and still treat it as temporary.
  6. Test with realistic callers: a user with narrow ACL groups, a small tenant, a date range that matches 0.1% — check result counts and recall.
  7. Monitor zero-result rate and recall on filtered canaries.

Small example

# Demonstrate the post-filter failure and the fix (Qdrant-style; same idea elsewhere)

def bad_post_filter(store, qvec, caller, k=10):
    hits = store.search("chunks", qvec, limit=100)                          # no filter
    mine = [h for h in hits if h.payload["tenant_id"] == caller.tenant_id and h.payload["is_current"]]
    return mine[:k]                                                         # often empty for small tenants

def good_filtered_ann(store, qvec, caller, k=10):
    flt = {"must": [{"key": "tenant_id", "match": {"value": caller.tenant_id}},
                    {"key": "is_current", "match": {"value": True}},
                    {"key": "acl_groups", "match": {"any": list(caller.groups)}}]}
    return store.search("chunks", qvec, query_filter=flt, limit=k * 3,
                        search_params={"hnsw_ef": 160, "exact": False})[:k]   # filter during traversal; higher ef for selective filters

# selectivity-aware exact fallback (when the filtered subset is tiny, brute force is both exact and fast)
def adaptive(store, qvec, caller, k=10):
    n_match = store.count("chunks", query_filter=build_filter(caller))     # cheap with payload indexes
    exact = n_match < 20_000
    return store.search("chunks", qvec, query_filter=build_filter(caller), limit=k, search_params={"exact": exact})

# pgvector: make sure the planner uses the HNSW index with the WHERE clause; check EXPLAIN ANALYZE
# SET hnsw.ef_search = 200;  -- and consider partial indexes per tenant for very large tenants

Pros (of native filtered ANN)

  • Correct result counts for selective filters; no empty pages for small tenants.
  • Security filters are enforced inside the engine, not in app code that can be bypassed.
  • Predictable latency across selectivity ranges when the engine switches strategies.

Cons

  • Engine-dependent behaviour: recall under selective filters varies widely; must be measured per store and version.
  • Higher ef/candidate counts raise latency for filtered queries.
  • Payload indexes cost memory and slow writes.
  • High-cardinality filters (ACL lists with thousands of groups) strain index structures; need group indirection (tenant / ACL capture at ingest).
  • pgvector's planner can still choose a sequential scan or post-filter path depending on statistics — needs EXPLAIN discipline.

When to use

Every multi-tenant or permission-aware system — which is nearly every enterprise system. Also any system with date/type filters that users rely on ("only current policies", "only Confluence").

When NOT to use (post-filter)

  • Never as the security mechanism. Acceptable only for non-security cosmetic filters over broad result sets (e.g. "hide already-shown results"), with over-fetch.

Production at scale

Selectivity profile per filter. Measure how selective each filter is in practice (tenant sizes follow a power law: a few huge, thousands tiny). Tiny tenants → exact search on their subset is fastest; huge tenants → filter-aware HNSW with tuned ef; the engine may do this automatically, but verify with EXPLAIN-equivalents or timing.

Partition the big ones. Physical collections/shards for the largest tenants remove the filter cost entirely and isolate their load; keep the tenant filter anyway (defence in depth).

ACL cardinality. acl_groups any-of caller_groups with callers in 200 groups and docs with 50 groups is a wide set intersection; use synthetic ACL-set IDs (tenant / ACL capture at ingest) to reduce cardinality, and index them as keywords.

is_current and version filters on every query (document versioning); the payload index on a boolean is cheap; some engines prefer you to delete old versions rather than filter — do both (filter now, GC later).

Recall canary with filters. Daily: golden queries executed as realistic callers (small tenant, restricted groups); recall vs filtered flat search; alert on drops after upgrades/compaction.

Zero-result handling. If a filtered query returns nothing, decide: widen non-security filters (dates), fall back to BM25 (hybrid search), or abstain (score thresholding & abstention). Never widen the security filter.

Latency budget. Filtered queries are slower; include them in the p95 SLO measurement rather than benchmarking unfiltered.

Cross-index consistency. BM25/sparse indexes must apply the same filters (hybrid search); fusing an ACL-filtered dense list with an unfiltered BM25 list leaks.

Prod checklist

  • Store with native filtered ANN; payload indexes on all filter fields
  • Security filters (tenant, ACL, current) built by one function, inside the engine query
  • ef/candidate counts tuned for filtered recall; adaptive exact search for tiny subsets
  • Physical partitioning for largest tenants; filter kept anyway
  • ACL cardinality controlled via synthetic set IDs
  • Filtered recall canary with realistic callers; zero-result rate monitored
  • Same filters applied to BM25/sparse index before fusion
  • pgvector: query plans verified with EXPLAIN after every schema/index change

Metrics to watch

MetricWhy
Filtered recall vs filtered flatEngine behaviour under selectivity
Zero-result rate by tenant size bucketPost-filter symptoms
Filtered p95 vs unfiltered p95Cost of filtering
Payload index memory; write latencyIndex cost