Generation & grounding → Faithfulness check (is each claim entailed?)

RAG Faithfulness Check Explained: Block Unsupported Claims

What it is

Citation validation (citations) proves an ID exists. It does not prove the cited passage says what the claim asserts. Faithfulness checking closes that gap: decompose the answer into atomic claims and, for each, ask whether the retrieved context entails it.

answer ──► split into atomic claims
              │
              ▼
        for each claim: entailed by cited context?
              ├── entailed          → keep
              ├── contradicted      → remove; flag (serious)
              └── not determinable  → remove, hedge, or regenerate
              │
              ▼
        aggregate faithfulness score → deliver / regenerate / abstain

Implementations range from cheap to thorough:

MethodCostLatencyQuality
NLI cross-encoder (DeBERTa-MNLI, vectara/hallucination_evaluation_model)Very low~10–50 msGood for short claims
Small LLM judge per claimLow–medium200–800 msBetter on nuance and numbers
Strong LLM judge on the whole answerHigh1–3 sBest; use offline or for high-stakes

Online (blocking, on the response path) versus offline (sampled, for monitoring) is a product decision: online prevents hallucinations reaching users at the cost of latency; offline measures them cheaply. Most systems do both — sampling everything offline, blocking only on high-stakes paths.

How to implement

  1. Get atomic claims. Best: the model already emits them (grounded prompting structured output). Fallback: sentence-split the answer and drop non-factual sentences.
  2. Pair each claim with its cited chunks (not the whole context) — checking against everything retrieved produces false "entailed" verdicts from unrelated passages.
  3. Score with an NLI model or a small LLM returning {verdict: entailed|contradicted|neutral, confidence}.
  4. Aggregate: faithfulness = entailed claims / factual claims. Set thresholds for deliver / hedge / regenerate / abstain.
  5. Act on the verdict — this is what makes it a control rather than a metric. Remove or rewrite unsupported claims, or regenerate once with an instruction naming the unsupported claim, or abstain.
  6. Special-case numbers and negations: NLI models are weak on numeric entailment and negation. Add a numeric check (does the figure appear in the cited chunk?) and treat negations with a stricter judge.
  7. Log everything: per-claim verdicts feed triage (feedback capture & triage) and the golden set (golden dataset first).

Small example

from transformers import pipeline
nli = pipeline("text-classification", model="MoritzLaurer/DeBERTa-v3-base-mnli-fever-anli", device=0)

NUM = re.compile(r"\b\d[\d,\.]*\s*(%|days?|months?|years?|usd|eur|gbp)?\b", re.I)

def numeric_support(claim: str, evidence: str) -> bool:
    nums = {n.group(0).strip().lower().replace(",", "") for n in NUM.finditer(claim)}
    if not nums: return True
    ev = evidence.lower().replace(",", "")
    return all(n in ev for n in nums)                       # every number in the claim appears in the evidence

def check_claim(claim: str, evidence: str) -> dict:
    if not numeric_support(claim, evidence):
        return {"verdict": "neutral", "score": 0.0, "reason": "number_not_in_evidence"}
    r = nli(f"{evidence[:1800]} [SEP] {claim}", truncation=True)[0]
    label = r["label"].lower()
    verdict = "entailed" if "entail" in label else "contradicted" if "contradict" in label else "neutral"
    return {"verdict": verdict, "score": float(r["score"])}

def faithfulness_gate(response: dict, retrieved: list[dict], tau_deliver=0.9, tau_regen=0.6):
    by_id = {c["chunk_id"]: c for c in retrieved}
    factual = [c for c in response["claims"] if is_factual(c["text"])]
    for cl in factual:
        evidence = "\n".join(by_id[cid]["text"] for cid in cl["citations"] if cid in by_id)
        cl["check"] = check_claim(cl["text"], evidence) if evidence else {"verdict": "neutral", "score": 0.0}
    supported = [c for c in factual if c["check"]["verdict"] == "entailed"]
    contradicted = [c for c in factual if c["check"]["verdict"] == "contradicted"]
    score = len(supported) / len(factual) if factual else 1.0
    log_faithfulness(score, n_claims=len(factual), n_contradicted=len(contradicted))

    if contradicted:                                        # never deliver a contradicted claim
        return regenerate_without(response, contradicted, retrieved)
    if score >= tau_deliver:
        return response
    if score >= tau_regen:
        return regenerate_with_feedback(response, [c for c in factual if c["check"]["verdict"] != "entailed"], retrieved)
    return abstain_with_partial(response, supported)

Pros

  • Catches hallucinations before delivery — the only control in the stack that operates on the generated text itself.
  • NLI models are cheap and fast enough to run on every response (tens of milliseconds).
  • Produces a per-response quality score usable for hedging, escalation, and monitoring.
  • Per-claim verdicts are excellent triage data — they point at the exact sentence and the exact passage.
  • Works regardless of model, prompt, or retrieval strategy.

Cons

  • False positives on paraphrase and inference: a correct claim that combines two passages may be judged "neutral" by a sentence-level NLI model.
  • Weak on numbers, dates, negations, and quantifiers — precisely the claims that matter most. Needs supplementary checks.
  • Latency on the blocking path (50 ms for NLI, up to seconds for LLM judges).
  • Regeneration doubles cost for the responses that fail.
  • Claim decomposition is itself error-prone if the model doesn't emit claims natively.
  • An entailment checker can be fooled by a plausible-but-wrong passage — it verifies consistency with retrieved text, not truth.

When to use

  • Offline sampling: always. It's the primary generation-quality metric (generation evals) and costs little.
  • Online blocking: high-stakes domains (legal, medical, financial, compliance), or any surface where a wrong fact has real consequences.
  • Whenever the golden set shows citation-valid but unsupported claims.

When NOT to use

  • Online, on latency-critical consumer paths with low stakes — sample offline instead.
  • As the sole hallucination control: it pairs with grounded prompting (grounded prompting), citation validation (citations), and retrieval thresholds (score thresholding & abstention).
  • On creative/drafting surfaces where the output is deliberately not grounded.

Production at scale

Check against the cited chunks, not the whole context. Checking a claim against all eight retrieved passages inflates entailment (something in there is usually loosely related) and defeats the purpose.

Numbers deserve their own check. A regex-based "does this figure appear in the evidence" check catches the most damaging hallucination class — invented amounts, dates, and percentages — for effectively zero cost, and it's more reliable than NLI on exactly these cases.

Two-tier verification: NLI on every response (cheap, blocking); a strong LLM judge on a 1–5% sample plus everything that NLI flagged (expensive, offline). Calibrate the NLI thresholds against the LLM judge and against human labels quarterly (LLM-as-judge).

Act, don't just measure. Define the policy explicitly: contradicted claims are never delivered; below tau_regen, regenerate once with the unsupported claims named; below that, abstain with the supported subset. Log every action.

Regeneration is bounded to one attempt — a second failure means abstain, not a third try (cost & step budgets discipline).

Hedging as a middle path: for scores between thresholds, deliver with an explicit confidence note rather than silently. Users tolerate "I'm fairly confident, based on…" far better than a confidently wrong answer.

Approximation-aware checking: VLM-derived chart descriptions (charts, images, diagrams) contain approximate values marked as such; the checker must not flag "approximately $4.2M" as unfaithful to "approximately 4.2". Propagate the approximation flag.

Monitor drift. Faithfulness by prompt version, model version, tenant, and query class, weekly. A drop after a model upgrade is one of the most common silent regressions (versioning & rollout).

Cost: a self-hosted NLI model on a small GPU handles thousands of checks per second; this is one of the cheapest quality controls available.

Prod checklist

  • Atomic claims from structured generation; sentence-split fallback
  • Each claim checked against its cited chunks only
  • Numeric/date support check in addition to NLI
  • Explicit action policy: contradicted → never deliver; low score → regenerate once → abstain
  • Two-tier verification: NLI online, LLM judge on samples and flagged cases
  • Thresholds calibrated against human labels; recalibrated on model changes
  • Approximation flags propagated from VLM-derived content
  • Per-claim verdicts logged and routed into triage and golden-set growth
  • Faithfulness tracked by prompt/model version, tenant, and query class

Metrics to watch

MetricWhy
Faithfulness score distributionCore generation quality
Contradicted-claim rateMost serious failures
Regeneration rate and its success rateControl effectiveness and cost
NLI vs LLM-judge agreementChecker calibration
Faithfulness by model/prompt versionRegression detection