Advanced RAG → Self-RAG (reflection tokens: retrieve? supported? useful?)
What it is
Self-RAG (Asai et al., 2023) trains a model to produce reflection tokens alongside its output:
| Token | Question it answers | Values |
|---|---|---|
Retrieve | Do I need to look something up for this segment? | yes / no / continue |
IsRel | Is this retrieved passage relevant? | relevant / irrelevant |
IsSup | Is this sentence I just wrote supported by the passage? | fully / partially / no support |
IsUse | Is this response useful for the question? | 1–5 |
Generation becomes segment-by-segment: decide whether to retrieve, retrieve if needed, generate a segment, self-assess support, and (in the original method) beam-search over candidate continuations ranked by the reflection scores. The model can answer from parametric knowledge when retrieval isn't needed, and it self-filters unsupported claims as it writes.
In practice, most teams implement a prompted approximation rather than the trained model: a structured output where the model emits needs_retrieval, per-claim support labels, and citations, validated in code. You lose the trained calibration; you keep the architecture and can use any model.
How to implement (the practical version)
- Adaptive retrieval decision. A cheap first call (or the router, routing) outputs
{"needs_retrieval": bool, "reason": ...}. Skip retrieval for chit-chat, formatting requests, and general knowledge — this is where much of Self-RAG's real-world value lies. - Relevance self-check on retrieved passages — in practice, use the reranker score (reranking) rather than asking the model; it's cheaper and better calibrated.
- Structured generation with per-claim support: the model outputs an array of
{claim, citations: [chunk_ids], support: "full"|"partial"|"none"}plus a final prose answer assembled from the claims. - Validate in code: every cited ID must exist in the retrieved set; claims labelled
noneare dropped or flagged; if too many claims are unsupported, abstain (score thresholding & abstention). - Optional usefulness pass: a separate cheap call scoring the assembled answer against the question; low scores trigger a retry or escalation (routing & escalation).
- Never trust the labels alone — sample-verify with an independent faithfulness check (faithfulness check); self-assessment is correlated with, not equal to, correctness.
- Log the reflection outputs — they're a rich per-claim quality signal for triage (feedback capture & triage).
Small example
SELF_RAG_SYS = """Answer the question using the passages. Work claim by claim.
For each factual claim you make, cite the passage ids that support it and label the support level.
Output JSON:
{
"needs_more_retrieval": bool,
"missing": string|null,
"claims": [{"text": string, "citations": [string], "support": "full"|"partial"|"none"}],
"answer": string,
"usefulness": 1-5
}
Rules:
- support="full" only when a cited passage states the claim explicitly.
- Do not make claims you cannot cite; if the passages lack the information, say so in "missing".
- "answer" must contain only claims labelled full or partial."""
def self_rag_answer(question, caller, max_retrieval_rounds=2):
evidence, rounds = [], 0
needs = adaptive_retrieval_decision(question) # cheap call or router
while rounds < max_retrieval_rounds:
if needs:
new = rerank_hosted(question, hybrid_retrieve(question, caller), top_n=6)
evidence = mmr_select(evidence + new, embed_queries([question])[0], k=8)
out = json.loads(llm("self_rag", SELF_RAG_SYS, [{"role": "user",
"content": f"<question>{question}</question>\n<passages>{render(evidence)}</passages>"}],
max_tokens=1200).text)
valid_ids = {c["chunk_id"] for c in evidence}
for cl in out["claims"]: # code validation, not self-report alone
bad = [c for c in cl["citations"] if c not in valid_ids]
if bad: cl["support"] = "none"; cl["invalid_citations"] = bad
unsupported = [c for c in out["claims"] if c["support"] == "none"]
log_self_rag(question, n_claims=len(out["claims"]), n_unsupported=len(unsupported), usefulness=out["usefulness"])
if out.get("needs_more_retrieval") and out.get("missing") and rounds + 1 < max_retrieval_rounds:
question_aug, needs, rounds = out["missing"], True, rounds + 1
continue
break
if not out["claims"] or len(unsupported) > len(out["claims"]) / 2:
return abstain(question, evidence)
return {"answer": out["answer"], "claims": [c for c in out["claims"] if c["support"] != "none"],
"usefulness": out["usefulness"]}
Pros
- Adaptive retrieval — skipping retrieval when it isn't needed saves latency and cost and avoids injecting irrelevant context into simple turns.
- Per-claim support labels make hallucinations visible at the sentence level rather than the answer level.
- Self-filtering removes unsupported claims before the user sees them.
- Citations become structural, which makes validation in code straightforward (citations).
- The reflection outputs are excellent triage data.
Cons
- Self-assessment is not reliable. A model that hallucinated a claim will often also label it supported. Treat labels as a signal, verify independently.
- The trained version requires a fine-tuned model with the reflection vocabulary; the prompted version is an approximation with weaker calibration.
- Token cost: structured per-claim output is substantially more tokens than prose.
- Latency: multiple passes if you add a usefulness check or a second retrieval round.
- Segment-level beam search (the paper's full method) is impractical behind most inference APIs.
When to use
- Mixed-traffic assistants where many turns don't need retrieval at all (adaptive retrieval is the biggest practical win).
- High-stakes answers where per-claim citation and support labelling is a product requirement.
- As a structured alternative to a separate post-hoc faithfulness pass, when you want the filtering inside generation.
When NOT to use
- Latency-critical single-purpose Q&A — a simpler pipeline plus a post-hoc faithfulness check (faithfulness check) is cheaper and about as effective.
- As your only hallucination control. Independent verification is required; self-labels are correlated, not conclusive.
- When output token cost dominates your bill — per-claim JSON is expensive.
Production at scale
Take the adaptive-retrieval half, be sceptical of the self-grading half. In practice the reliable value of Self-RAG in production systems is the "should I retrieve at all" decision — often 20–40% of conversational turns — and the structured per-claim citation format. The self-support labels are useful as a cheap filter whose output you then verify on a sample.
Validate citations in code, always. Every cited ID must exist in the retrieved set; invalid citations force support: none. This single check catches most fabricated citations and costs nothing (citations).
Calibrate the labels. On a sample, compare model-assigned support against human or NLI judgement (faithfulness check). Publish the confusion matrix. If full means "actually supported" only 80% of the time, your product copy shouldn't say "verified".
Cost control: per-claim JSON roughly doubles output tokens versus prose. Use it on the high-stakes path only, or emit claims for the citation-bearing sentences and prose for the rest.
Combine, don't duplicate. If you already run CRAG-style grading (corrective RAG (CRAG): grade, then rewrite or fall back) before generation and a faithfulness check (faithfulness check) after, adding full Self-RAG in the middle is three overlapping controls. Pick the two that measure best on your eval set.
Abstention rule from claim labels: if more than half the claims are unsupported, abstain rather than presenting a filtered stub. Log it as a distinct abstention reason.
Reflection outputs into the flywheel (data flywheel): claims labelled partial or none are precisely the cases worth reviewing and turning into golden examples or hard negatives.
Prod checklist
- Adaptive retrieval decision implemented (the highest-value half)
- Structured per-claim output with citations and support labels
- Citation IDs validated against the retrieved set in code; invalid →
none - Abstention rule when unsupported claims dominate
- Independent faithfulness verification on a sample; label calibration published
- Bounded extra retrieval rounds (≤2)
- Per-claim output reserved for high-stakes paths (token cost)
- Reflection outputs routed into triage and golden-set growth
Metrics to watch
| Metric | Why |
|---|---|
| Adaptive-retrieval skip rate | Cost/latency saved |
| Claim support-label distribution | Generation health |
| Label calibration (self-label vs verified) | How much to trust it |
| Invalid-citation rate | Fabrication |
| Abstention rate from unsupported-claim rule | Safety net firing |
Related
- Corrective RAG — grading before generation instead of during
- Citations — the validation this depends on
- Faithfulness check — the independent verifier
- Routing — an easier place to make the retrieve/don't-retrieve call
- Data flywheel