Query understanding → Self-query (natural language → metadata filters)
What it is
Embeddings are bad at constraints. Dates, numeric ranges, enumerated categories, and negations ("not the draft version") are structural facts that vector similarity approximates poorly — a 2019 contract and a 2024 contract embed almost identically. But your chunks already carry that metadata (metadata on every chunk). Self-query uses an LLM to parse the user's question into:
{
"semantic_query": "vendor termination clauses",
"filters": {"signed_date": {"gte": "2023-01-01"}, "region": {"eq": "EMEA"}, "doc_type": {"eq": "contract"}}
}
The semantic part goes to the retriever; the filters go to the vector store's filter clause (filtered search), combined with the mandatory security filters. The result: precision that dense retrieval cannot achieve, and result counts the user can trust ("47 contracts match").
How to implement
- Publish the filterable schema to the LLM: field names, types, allowed enum values, and date semantics. This is a closed vocabulary — the model must not invent fields or values.
- Extraction prompt (fast tier, temperature 0) with the schema and 3–4 examples, including one with no filters and one with an ambiguous phrase that should not become a filter.
- Validate hard: unknown field → drop; enum value not in the allowed list → drop (or fuzzy-map to the nearest allowed value with a confidence threshold); malformed date → drop. Never pass an unvalidated filter to the store.
- Resolve relative dates ("last quarter", "since the merger") in code against a reference date passed into the prompt, not by the model's arithmetic.
- Merge with security filters — self-query filters are ANDed on top of tenant/ACL/current-version filters, never replacing them (tenant / ACL capture at ingest).
- Handle empty results: if filters yield zero, retry without the non-security filters and tell the user which constraint was relaxed. Never relax security filters.
- Evaluate: filter extraction precision/recall against a labelled set of queries, plus end-to-end retrieval quality.
Small example
from datetime import date
from pydantic import BaseModel, field_validator
FILTER_SCHEMA = {
"doc_type": {"type": "enum", "values": ["contract", "policy", "invoice", "report", "email"]},
"region": {"type": "enum", "values": ["EMEA", "AMER", "APAC"]},
"signed_date": {"type": "date"},
"vendor": {"type": "string"},
"status": {"type": "enum", "values": ["draft", "executed", "expired"]},
}
SELFQ_SYS = f"""Extract search filters from the user's question. Today is {{today}}.
Available fields (use ONLY these, with ONLY these values):
{json.dumps(FILTER_SCHEMA, indent=2)}
Operators: eq, in, gte, lte, ne. Resolve relative dates to absolute ISO dates using today's date.
If a phrase does not clearly map to a field, leave it in semantic_query instead of inventing a filter.
Output JSON: {{"semantic_query": string, "filters": {{field: {{op: value}}}}}}
<example>
<q>termination clauses in contracts signed after 2023 with EMEA vendors</q>
<out>{{"semantic_query": "termination clauses", "filters": {{"doc_type": {{"eq": "contract"}}, "signed_date": {{"gte": "2023-01-01"}}, "region": {{"eq": "EMEA"}}}}}}</out>
</example>
<example>
<q>what is our parental leave policy</q>
<out>{{"semantic_query": "parental leave policy", "filters": {{}}}}</out>
</example>"""
ALLOWED_OPS = {"eq", "in", "gte", "lte", "ne"}
def validate_filters(raw: dict) -> tuple[dict, list[str]]:
clean, dropped = {}, []
for field, cond in (raw or {}).items():
spec = FILTER_SCHEMA.get(field)
if not spec: dropped.append(f"unknown field {field}"); continue
for op, val in cond.items():
if op not in ALLOWED_OPS: dropped.append(f"{field}: bad op {op}"); continue
if spec["type"] == "enum" and val not in spec["values"]:
dropped.append(f"{field}: value {val!r} not allowed"); continue
if spec["type"] == "date" and not re.fullmatch(r"\d{4}-\d{2}-\d{2}", str(val)):
dropped.append(f"{field}: bad date {val!r}"); continue
clean.setdefault(field, {})[op] = val
return clean, dropped
def self_query(question: str, caller):
out = llm("self_query", SELFQ_SYS.format(today=date.today().isoformat()),
[{"role": "user", "content": question}], max_tokens=250)
parsed = json.loads(out.text)
filters, dropped = validate_filters(parsed.get("filters"))
log_self_query(question, filters, dropped)
combined = merge_filters(build_acl_filter(caller), filters) # security filters always win and are never removable
hits = vec.search(embed_queries([parsed["semantic_query"]])[0], k=30, filter=combined)
if not hits and filters: # relax NON-security filters only
hits = vec.search(embed_queries([parsed["semantic_query"]])[0], k=30, filter=build_acl_filter(caller))
return hits, {"relaxed": list(filters)}
return hits, {"applied": filters, "dropped": dropped}
Pros
- Precision on constraints embeddings can't express: dates, ranges, categories, negation.
- Trustworthy counts and scoping — "search only executed contracts" actually does that.
- Smaller, cleaner candidate sets → reranking and generation both improve.
- Enables faceted UX (show the parsed filters as removable chips — users love this and it makes the system's reasoning visible).
Cons
- Hallucinated filters silently exclude the right documents — the worst failure here, because the user sees a confident "no results" or a wrong subset.
- Schema coupling: the prompt must stay in sync with the actual metadata fields and enum values.
- Over-filtering on ambiguous phrases ("recent", "big") that shouldn't have become filters.
- Selectivity interaction: adding filters to an ANN query changes recall behaviour (filtered search).
- Another LLM call on the critical path (fold into the rewrite call).
When to use
- Corpora with rich, reliable metadata and users who ask scoped questions (legal, finance, compliance, procurement, ticketing).
- Any UI where users expect filters but type sentences.
- When retrieval evals show failures caused by returning the wrong version, date, or type of an otherwise-relevant document.
When NOT to use
- Sparse or unreliable metadata — filters on fields that are often null exclude good documents. Fix ingestion first (metadata on every chunk).
- Purely semantic corpora (wiki prose with no structured attributes).
- When the user's phrasing rarely contains constraints — measure the firing rate before building it.
Production at scale
Show the filters, let users remove them. Rendering parsed filters as chips is the single best mitigation for hallucinated filters: the user sees "region: EMEA" and clicks it off. It converts a silent failure into a visible, correctable one.
Enum grounding from the index, not the prompt. Generate the allowed-values list from the actual distinct values in the corpus (per tenant, refreshed periodically) rather than hardcoding it. A prompt claiming region ∈ {EMEA, AMER, APAC} while the data says emea/EU produces filters that match nothing.
Fuzzy value mapping with a threshold: "Europe" → EMEA only if a similarity/alias table says so; otherwise leave it in the semantic query. Maintain the alias table per tenant.
Never relax security filters on empty results — only user-intent filters, and say which one was relaxed. Log every relaxation.
Zero-result loop: filters → 0 results → relax → still 0 → abstain (score thresholding & abstention). Track the zero-result rate by whether filters were applied; a high rate with filters means extraction is too aggressive or metadata coverage is poor.
Fold into one call. Rewrite (query rewriting) + decomposition detection (decomposition) + filter extraction in a single fast-tier structured output saves 2 round trips (structured outputs & tool use).
Metadata coverage dashboard. Percentage of chunks with each filterable field populated, per source. Self-query on a field that's 40% null is a trap.
Evaluation set of (query → expected filters) pairs, scored on precision and recall per field. Filter precision matters more than recall: a wrong filter excludes the answer; a missing filter just means a broader search that reranking can still fix.
Prod checklist
- Filterable schema generated from actual index values, per tenant, refreshed
- Extraction validated against the closed schema; unknown fields/values dropped and logged
- Relative dates resolved in code against a passed reference date
- Security filters merged in and never relaxable
- Parsed filters surfaced in the UI as removable chips
- Zero-result relaxation path (non-security only) with user-visible explanation
- Folded into the rewrite call to save a round trip
- Metadata coverage per field per source monitored
- Labelled (query → filters) eval set; per-field precision tracked
Metrics to watch
| Metric | Why |
|---|---|
| Filter extraction precision per field | Hallucinated filters exclude answers |
| Zero-result rate with vs without filters | Over-filtering |
| Filter relaxation rate | Extraction aggressiveness |
| Metadata field coverage per source | Whether filtering is viable at all |
Related
- Chunk metadata — the fields this depends on
- Filtered search — how filters interact with ANN
- Tenant/ACL — security filters are separate and mandatory
- Query rewriting — share the call
- Abstention — when filters legitimately yield nothing