Generation & grounding → Faithfulness check (is each claim entailed?)
What it is
Citation validation (citations) proves an ID exists. It does not prove the cited passage says what the claim asserts. Faithfulness checking closes that gap: decompose the answer into atomic claims and, for each, ask whether the retrieved context entails it.
answer ──► split into atomic claims
│
▼
for each claim: entailed by cited context?
├── entailed → keep
├── contradicted → remove; flag (serious)
└── not determinable → remove, hedge, or regenerate
│
▼
aggregate faithfulness score → deliver / regenerate / abstain
Implementations range from cheap to thorough:
| Method | Cost | Latency | Quality |
|---|---|---|---|
NLI cross-encoder (DeBERTa-MNLI, vectara/hallucination_evaluation_model) | Very low | ~10–50 ms | Good for short claims |
| Small LLM judge per claim | Low–medium | 200–800 ms | Better on nuance and numbers |
| Strong LLM judge on the whole answer | High | 1–3 s | Best; use offline or for high-stakes |
Online (blocking, on the response path) versus offline (sampled, for monitoring) is a product decision: online prevents hallucinations reaching users at the cost of latency; offline measures them cheaply. Most systems do both — sampling everything offline, blocking only on high-stakes paths.
How to implement
- Get atomic claims. Best: the model already emits them (grounded prompting structured output). Fallback: sentence-split the answer and drop non-factual sentences.
- Pair each claim with its cited chunks (not the whole context) — checking against everything retrieved produces false "entailed" verdicts from unrelated passages.
- Score with an NLI model or a small LLM returning
{verdict: entailed|contradicted|neutral, confidence}. - Aggregate: faithfulness = entailed claims / factual claims. Set thresholds for deliver / hedge / regenerate / abstain.
- Act on the verdict — this is what makes it a control rather than a metric. Remove or rewrite unsupported claims, or regenerate once with an instruction naming the unsupported claim, or abstain.
- Special-case numbers and negations: NLI models are weak on numeric entailment and negation. Add a numeric check (does the figure appear in the cited chunk?) and treat negations with a stricter judge.
- Log everything: per-claim verdicts feed triage (feedback capture & triage) and the golden set (golden dataset first).
Small example
from transformers import pipeline
nli = pipeline("text-classification", model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli", device=0)
NUM = re.compile(r"\b\d[\d,\.]*\s*(%|days?|months?|years?|usd|eur|gbp)?\b", re.I)
def numeric_support(claim: str, evidence: str) -> bool:
nums = {n.group(0).strip().lower().replace(",", "") for n in NUM.finditer(claim)}
if not nums: return True
ev = evidence.lower().replace(",", "")
return all(n in ev for n in nums) # every number in the claim appears in the evidence
def check_claim(claim: str, evidence: str) -> dict:
if not numeric_support(claim, evidence):
return {"verdict": "neutral", "score": 0.0, "reason": "number_not_in_evidence"}
r = nli(f"{evidence[:1800]} [SEP] {claim}", truncation=True)[0]
label = r["label"].lower()
verdict = "entailed" if "entail" in label else "contradicted" if "contradict" in label else "neutral"
return {"verdict": verdict, "score": float(r["score"])}
def faithfulness_gate(response: dict, retrieved: list[dict], tau_deliver=0.9, tau_regen=0.6):
by_id = {c["chunk_id"]: c for c in retrieved}
factual = [c for c in response["claims"] if is_factual(c["text"])]
for cl in factual:
evidence = "\n".join(by_id[cid]["text"] for cid in cl["citations"] if cid in by_id)
cl["check"] = check_claim(cl["text"], evidence) if evidence else {"verdict": "neutral", "score": 0.0}
supported = [c for c in factual if c["check"]["verdict"] == "entailed"]
contradicted = [c for c in factual if c["check"]["verdict"] == "contradicted"]
score = len(supported) / len(factual) if factual else 1.0
log_faithfulness(score, n_claims=len(factual), n_contradicted=len(contradicted))
if contradicted: # never deliver a contradicted claim
return regenerate_without(response, contradicted, retrieved)
if score >= tau_deliver:
return response
if score >= tau_regen:
return regenerate_with_feedback(response, [c for c in factual if c["check"]["verdict"] != "entailed"], retrieved)
return abstain_with_partial(response, supported)
Pros
- Catches hallucinations before delivery — the only control in the stack that operates on the generated text itself.
- NLI models are cheap and fast enough to run on every response (tens of milliseconds).
- Produces a per-response quality score usable for hedging, escalation, and monitoring.
- Per-claim verdicts are excellent triage data — they point at the exact sentence and the exact passage.
- Works regardless of model, prompt, or retrieval strategy.
Cons
- False positives on paraphrase and inference: a correct claim that combines two passages may be judged "neutral" by a sentence-level NLI model.
- Weak on numbers, dates, negations, and quantifiers — precisely the claims that matter most. Needs supplementary checks.
- Latency on the blocking path (50 ms for NLI, up to seconds for LLM judges).
- Regeneration doubles cost for the responses that fail.
- Claim decomposition is itself error-prone if the model doesn't emit claims natively.
- An entailment checker can be fooled by a plausible-but-wrong passage — it verifies consistency with retrieved text, not truth.
When to use
- Offline sampling: always. It's the primary generation-quality metric (generation evals) and costs little.
- Online blocking: high-stakes domains (legal, medical, financial, compliance), or any surface where a wrong fact has real consequences.
- Whenever the golden set shows citation-valid but unsupported claims.
When NOT to use
- Online, on latency-critical consumer paths with low stakes — sample offline instead.
- As the sole hallucination control: it pairs with grounded prompting (grounded prompting), citation validation (citations), and retrieval thresholds (score thresholding & abstention).
- On creative/drafting surfaces where the output is deliberately not grounded.
Production at scale
Check against the cited chunks, not the whole context. Checking a claim against all eight retrieved passages inflates entailment (something in there is usually loosely related) and defeats the purpose.
Numbers deserve their own check. A regex-based "does this figure appear in the evidence" check catches the most damaging hallucination class — invented amounts, dates, and percentages — for effectively zero cost, and it's more reliable than NLI on exactly these cases.
Two-tier verification: NLI on every response (cheap, blocking); a strong LLM judge on a 1–5% sample plus everything that NLI flagged (expensive, offline). Calibrate the NLI thresholds against the LLM judge and against human labels quarterly (LLM-as-judge).
Act, don't just measure. Define the policy explicitly: contradicted claims are never delivered; below tau_regen, regenerate once with the unsupported claims named; below that, abstain with the supported subset. Log every action.
Regeneration is bounded to one attempt — a second failure means abstain, not a third try (cost & step budgets discipline).
Hedging as a middle path: for scores between thresholds, deliver with an explicit confidence note rather than silently. Users tolerate "I'm fairly confident, based on…" far better than a confidently wrong answer.
Approximation-aware checking: VLM-derived chart descriptions (charts, images, diagrams) contain approximate values marked as such; the checker must not flag "approximately $4.2M" as unfaithful to "approximately 4.2". Propagate the approximation flag.
Monitor drift. Faithfulness by prompt version, model version, tenant, and query class, weekly. A drop after a model upgrade is one of the most common silent regressions (versioning & rollout).
Cost: a self-hosted NLI model on a small GPU handles thousands of checks per second; this is one of the cheapest quality controls available.
Prod checklist
- Atomic claims from structured generation; sentence-split fallback
- Each claim checked against its cited chunks only
- Numeric/date support check in addition to NLI
- Explicit action policy: contradicted → never deliver; low score → regenerate once → abstain
- Two-tier verification: NLI online, LLM judge on samples and flagged cases
- Thresholds calibrated against human labels; recalibrated on model changes
- Approximation flags propagated from VLM-derived content
- Per-claim verdicts logged and routed into triage and golden-set growth
- Faithfulness tracked by prompt/model version, tenant, and query class
Metrics to watch
| Metric | Why |
|---|---|
| Faithfulness score distribution | Core generation quality |
| Contradicted-claim rate | Most serious failures |
| Regeneration rate and its success rate | Control effectiveness and cost |
| NLI vs LLM-judge agreement | Checker calibration |
| Faithfulness by model/prompt version | Regression detection |
Related
- Grounded prompting, Citations — the layers before this
- Refusal — the terminal action
- Score thresholding — the retrieval-side control
- LLM-as-judge, Generation evals
- VLM captioning — approximate values