Common failure modes (and the fix)

Video takeaway

"Eight symptoms cover most production AI failures, and each has a known fix. The right document exists but is never retrieved — that's dense-only search; add keyword search and contextual headers. It cites the wrong section — chunks too big, no reranker. Confidently wrong on a question your corpus can't answer — no abstention threshold. Correct on turn one, wrong on turn three — no query rewriting over history. Costs ten times the estimate — check your cache hit rate first. Table questions always wrong — the table got flattened at parse time. And when the agent does something destructive, the fix isn't a better prompt, it's a credential that couldn't do it. Look up the symptom, but confirm with the diagnostic ladder before you fix anything."

What it is

SymptomUsual causeFixFiles
Right doc exists, never retrievedChunk lacks context/keywords; dense-only searchHybrid + contextual retrieval + query rewritingHybrid search, contextual retrieval, query rewriting
Answer cites the wrong sectionChunks too large, no rerankingParent–child + cross-encoder rerankParent–child, reranking
Confidently wrong on a fresh questionNo abstention pathScore threshold + faithfulness checkScore thresholding & abstention, faithfulness check
Correct at turn 1, wrong at turn 3No query rewriting over historyCoreference-resolving rewrite stepQuery rewriting, compaction / summarisation of long history
Costs 10× the estimateNo caching, top-k too large, agent loopsPrompt cache, rerank-then-trim, step budgetsPrompt caching of static prefixes, reranking, cost & step budgets
Works in dev, fails in prodEval set isn't representativeSample real traffic into the golden setGolden dataset first, online eval, data flywheel
Table questions always wrongTable flattened at parse timeStructure-preserving parser, table-aware chunksTables, complex layout, layout / structure-aware
Agent takes a destructive actionExcessive agency + injectionLeast-privilege tokens, HITL gate, data≠instructionsExcessive agency, human-in-the-loop, prompt-injection defence

Four more that recur often enough to add:

SymptomUsual causeFixFiles
Answers a superseded policyOld versions still retrievableVersion filter + is_current on every queryDocument versioning, conflict handling
Top-5 results are five copies of one paragraphNo dedupe; chunk overlapExact/near dedupe at ingest + MMR at query timeDeduplication, MMR / diversity
Refuses things it clearly knowsThreshold too strict; over-grounded promptRecalibrate τ; measure false-refusal rateScore thresholding & abstention, grounded prompting
Quality dropped and nobody knows whyNo version vector in traces; unpinned modelVersion-stamped traces; pin modelsObservability & tracing, versioning & rollout

How to implement (the diagnosis workflow)

  1. Reproduce with the trace (observability & tracing): the query, the caller, the retrieved chunk IDs and scores, the version vector.
  2. Run the diagnostic ladder (component vs end-to-end) to find the first failing stage — this maps directly to a row above.
  3. Confirm with a second case of the same shape; a single instance can be an outlier.
  4. Check the version vector: did anything change (prompt, model, index, parser) around when it started?
  5. Apply the fix at the identified stage, not at the stage where the symptom appeared.
  6. Add the case to the golden set and confirm it fails before the fix and passes after (data flywheel).
  7. Check whether the same cause explains other open failures — most of these are systemic, not isolated.

Small example

SYMPTOM_PLAYBOOK = {
    "right_doc_never_retrieved": {
        "check": lambda t: t["expected_chunk_ids"] and not set(t["expected_chunk_ids"]) & set(t["pool_ids"]),
        "diagnostics": ["is the query identifier-heavy? → is BM25 in the mix (hybrid search)",
                        "do the chunks carry breadcrumbs/context headers? (layout / structure-aware, contextual retrieval)",
                        "was the query a pronoun-laden follow-up? (query rewriting)",
                        "did an ACL or date filter exclude it? (filtered search, self-query)"],
        "fix_order": ["hybrid search", "contextual retrieval", "query rewriting", "embedding fine-tune"],
    },
    "cites_wrong_section": {
        "check": lambda t: t["expected_chunk_ids"] and set(t["expected_chunk_ids"]) & set(t["pool_ids"])
                           and not set(t["expected_chunk_ids"]) & set(t["final_ids"]),
        "diagnostics": ["is a cross-encoder reranker in place? (reranking)",
                        "are chunks too large to be precise? (parent–child)"],
        "fix_order": ["add reranking", "parent-child chunking", "reduce child chunk size"],
    },
    "confidently_wrong_fresh_question": {
        "check": lambda t: t.get("retrieval_top_score", 1) < 0.35 and not t.get("abstained"),
        "diagnostics": ["is there a score threshold? (score thresholding & abstention)",
                        "is there a faithfulness gate? (faithfulness check)",
                        "does the golden set contain unanswerable questions? (golden dataset first)"],
        "fix_order": ["score threshold + abstention", "faithfulness gate", "grounded prompt rules"],
    },
    "cost_blowup": {
        "check": lambda t: t["cost_usd"] > 10 * MEDIAN_COST,
        "diagnostics": ["cache hit rate on this call site? (prompt caching of static prefixes)",
                        "how many chunks entered the context? (reranking, per-source truncation budgets)",
                        "agent step count and budget? (cost & step budgets)",
                        "escalation rate to the reasoning tier? (routing & escalation)"],
        "fix_order": ["prompt caching", "rerank-then-trim", "step/token budgets", "routing thresholds"],
    },
    "destructive_agent_action": {
        "check": lambda t: any(a["writes"] and not a.get("approved") for a in t.get("actions", [])),
        "diagnostics": ["does the caller's role even permit this tool? (excessive agency)",
                        "was flagged content in context? (prompt-injection defence)",
                        "is there an approval gate for this tool? (human-in-the-loop)"],
        "fix_order": ["remove the tool from the role", "tainted-context suppression", "approval gate"],
        "severity": "critical",
    },
}

def diagnose(trace) -> list[dict]:
    """Match a failing trace against the playbook; ordered by confidence."""
    matches = []
    for name, spec in SYMPTOM_PLAYBOOK.items():
        try:
            if spec["check"](trace):
                matches.append({"symptom": name, "diagnostics": spec["diagnostics"],
                                "fix_order": spec["fix_order"],
                                "severity": spec.get("severity", "normal")})
        except (KeyError, TypeError):
            continue
    return sorted(matches, key=lambda m: 0 if m["severity"] == "critical" else 1)

def systemic_check(symptom: str, window_days=7) -> dict:
    """Is this one bug or a class? Almost always a class."""
    traces = failing_traces(window_days)
    matching = [t for t in traces if SYMPTOM_PLAYBOOK[symptom]["check"](t)]
    return {"count": len(matching), "share": len(matching) / max(len(traces), 1),
            "concentrated_in": Counter(t["source"] for t in matching).most_common(3),
            "verdict": "systemic" if len(matching) / max(len(traces), 1) > 0.15 else "isolated"}

Pros

  • Fast diagnosis: a symptom lookup turns hours of investigation into minutes.
  • Prevents fixing the wrong layer — the most common waste in AI projects is prompt-tuning a retrieval bug.
  • Shared vocabulary across a team, and a natural onboarding document.
  • Each row links to a tested remedy rather than a general principle.

Cons

  • Symptoms overlap: "wrong answer" can match several rows; the ladder (component vs end-to-end) disambiguates, the table alone does not.
  • Playbook thinking can stop investigation early on a genuinely novel failure.
  • Fixes have costs: reranking adds latency, parent–child adds complexity — the table names the remedy, not the trade-off.
  • Needs trace data to apply; without chunk IDs and scores, most checks can't run (observability & tracing).

When to use

Whenever something is wrong — as the first lookup after reproducing with a trace. Also as a design checklist: a system with none of these mitigations will exhibit most of these symptoms.

When NOT to use

  • As a substitute for the diagnostic ladder when the symptom is ambiguous.
  • For novel or safety-critical failures, where a full investigation is warranted regardless of pattern match.

Production at scale

Most of these are systemic, not isolated. When one appears, run the systemic check: how many of the last week's failures share it, and are they concentrated in one source, tenant, or document type? Fixing the class is usually one change; fixing instances is endless (feedback capture & triage).

Two of these are prevention, not diagnosis. "Agent takes a destructive action" and "answers a superseded policy" should be structurally impossible — least-privilege credentials (excessive agency) and version filtering (document versioning) — rather than symptoms you fix after they occur.

"Works in dev, fails in prod" is an eval-set problem, and the fix is the flywheel (data flywheel): continuous sampling of real traffic into the golden set. If this symptom recurs, the loop isn't running.

"Quality dropped and nobody knows why" is an observability problem. The version vector on every trace (observability & tracing) and pinned models (versioning & rollout) turn it into a five-minute bisection.

Cost blowups have four common causes in this order: cache hit rate collapse (a variable in the prompt prefix, prompt caching of static prefixes), top-k too large (reranking), unbounded agent loops (cost & step budgets), and over-escalation to the reasoning tier (routing & escalation). Check them in that order.

Track symptom frequency over time as a companion to the failure-bucket histogram (feedback capture & triage) — it shows which mitigations have landed.

Add your own rows. Every domain develops its own recurring failures; the table is a starting set, not a closed list, and extending it is how team knowledge accumulates.

Prod checklist

  • Trace-based reproduction available for any reported failure (chunk IDs, scores, version vector)
  • Diagnostic ladder run before applying a fix
  • Systemic check performed: is this a class or an instance?
  • Fix applied at the diagnosed stage, not the symptom stage
  • Case added to the golden set; fails before, passes after
  • Structural preventions in place for the two critical rows (least privilege, version filtering)
  • Symptom frequency tracked over time
  • Playbook extended with domain-specific failures as they're found

Metrics to watch

MetricWhy
Symptom frequency by type over timeWhich mitigations have landed
Systemic vs isolated ratioWhether you're fixing classes or instances
Median time from report to diagnosisObservability effectiveness
Recurrence of previously-fixed symptomsDurability of fixes