Retrieval & ranking → MMR / diversity

Maximal Marginal Relevance (MMR) in RAG Explained: Fix Duplicate Chunks

What it is

Relevance ranking is greedy: if one paragraph is highly relevant, its overlapping neighbour and the same paragraph from a duplicate document are also highly relevant. A top-5 can easily be five views of one fact — and the model then sees one fact where the user asked a question needing three.

Maximal Marginal Relevance re-selects the list iteratively:

MMR(d) = λ · relevance(q, d) − (1 − λ) · max  similarity(d, d')
                                        d' ∈ already-selected

λ = 1.0 → pure relevance (no diversity)
λ = 0.7 → mild diversity (a good default)
λ = 0.5 → strong diversity

At each step, pick the candidate with the best combination of "relevant to the query" and "different from what I've already picked". Similarity is usually cosine between chunk embeddings (already available) or a cheap lexical overlap.

Sources of redundancy MMR fixes: chunk overlap (fixed-size + overlap), duplicate documents that dedupe missed (deduplication), boilerplate repeated across documents (headers / footers / boilerplate stripping), multiple versions, and multi-query fusion returning the same chunk via different variants (multi-query + Reciprocal Rank Fusion).

How to implement

  1. Run after reranking, on the reranked candidate list (say top-30), selecting the final 5–8. Use the reranker score as relevance and chunk embeddings for similarity.
  2. Pick λ by evaluation: 0.7 is a good starting point. Multi-fact and comparison questions want lower λ; single-fact lookups want higher.
  3. Cheaper alternatives that often suffice: cap chunks per document (max 2), cap chunks per parent section, and drop chunks whose normalised text hash matches one already selected.
  4. Normalise scores before combining — reranker scores and cosine similarities are on different scales.
  5. Log the diversity applied: how many candidates were dropped for redundancy, and their similarity to the selected item.
  6. Evaluate on multi-fact golden questions specifically — MMR's value is invisible on single-fact questions and shows up strongly on "list all", "compare", and "what are the exceptions" queries.

Small example

import numpy as np

def mmr_select(candidates: list[dict], query_vec: np.ndarray, k: int = 6, lam: float = 0.7) -> list[dict]:
    """candidates: reranked, each with 'embedding' (unit norm) and 'rerank_score'."""
    if not candidates: return []
    rel = np.array([c["rerank_score"] for c in candidates], dtype=float)
    rel = (rel - rel.min()) / ((rel.max() - rel.min()) or 1.0)          # normalise to [0,1]
    E = np.stack([c["embedding"] for c in candidates])
    selected, remaining = [], list(range(len(candidates)))
    while remaining and len(selected) < k:
        if not selected:
            best = int(np.argmax(rel[remaining])); chosen = remaining[best]
        else:
            sim_to_sel = E[remaining] @ E[selected].T                    # cosine, vectors are unit norm
            redundancy = sim_to_sel.max(axis=1)
            score = lam * rel[remaining] - (1 - lam) * redundancy
            chosen = remaining[int(np.argmax(score))]
        selected.append(chosen); remaining.remove(chosen)
    return [candidates[i] for i in selected]

# Cheaper heuristics that cover most of the benefit
def cap_per_document(candidates, k=6, max_per_doc=2, max_per_parent=1):
    out, per_doc, per_parent = [], {}, {}
    for c in candidates:                                                 # already in relevance order
        d, p = c["doc_id"], c.get("parent_id")
        if per_doc.get(d, 0) >= max_per_doc: continue
        if p and per_parent.get(p, 0) >= max_per_parent: continue
        out.append(c); per_doc[d] = per_doc.get(d, 0) + 1
        if p: per_parent[p] = per_parent.get(p, 0) + 1
        if len(out) == k: break
    return out

Pros

  • More facts per token — the context budget covers distinct information rather than repetitions.
  • Better multi-fact and comparison answers, and better coverage of exceptions and edge cases buried in different documents.
  • Cheap: cosine over embeddings you already have; microseconds.
  • Masks residual failures in dedupe (deduplication) and overlap (fixed-size + overlap) without re-ingesting.

Cons

  • Can drop a genuinely needed near-duplicate: two contract versions that differ in one clause look similar but the difference is the answer. λ too low actively harms these.
  • λ is query-type dependent — one global value is a compromise.
  • Similarity ≠ redundancy: two chunks about the same topic from different jurisdictions are similar but not redundant.
  • Adds a tuning knob and a failure mode that's hard to see (the dropped chunk is invisible in the answer).

When to use

  • Corpora with known redundancy: overlapping chunks, duplicated documents, templated content, versioned policies.
  • Multi-fact question types ("list", "compare", "what are all the…").
  • After multi-query fusion, which structurally increases duplicate hits.

When NOT to use

  • Single-fact factoid lookups where you want the single best passage and its immediate neighbours — diversity actively hurts.
  • Corpora already well-deduped with structure-aware chunking and no overlap — measure the duplicate rate in top-k before adding MMR.
  • When the "duplicates" are versions and version disambiguation is the point — fix with version filters (document versioning), not diversity.

Production at scale

Prefer structural caps to MMR where you can. "Max 2 chunks per document, max 1 per parent section" is deterministic, explainable, and free — and covers most redundancy in practice. Reach for MMR when redundancy is cross-document and semantic rather than structural.

Query-type-aware λ. If routing or decomposition (decomposition/04.4.6) already classifies the query, use λ≈0.85 for single-fact lookups and λ≈0.6 for list/compare questions. Small change, measurable gain.

Measure the duplicate rate first. Instrument "mean pairwise cosine among final top-k" and "distinct doc_ids in top-k". If pairwise similarity is already low, MMR has nothing to do and you're adding a risk for no gain.

Never diversify away the top-1. The highest-relevance chunk is always selected first; MMR only affects positions 2..k. Assert this.

Interaction with parent–child. Diversify at the child level before collapsing to parents, or collapse first and diversify parents — collapsing already removes much redundancy, so MMR after collapse usually needs a higher λ.

Explainability. Log which candidates were suppressed and by what similarity to which selected chunk. When a user says "you missed the section on X", this log tells you whether retrieval missed it or diversity dropped it — a distinction that otherwise takes hours to untangle.

Evaluation split. Report answer quality separately for single-fact and multi-fact golden questions. A global average can show "no change" while MMR is helping one class and hurting the other.

Prod checklist

  • Applied after reranking, selecting final k from a reranked pool
  • Top-1 always selected regardless of diversity
  • Structural caps (per document, per parent) evaluated before/alongside MMR
  • λ tuned on the golden set; query-type-aware if classification exists
  • Scores normalised before combining relevance and similarity
  • Suppressed candidates logged with the reason and the similar selected chunk
  • Duplicate-rate telemetry in top-k (mean pairwise similarity, distinct docs)
  • Eval reported separately for single-fact vs multi-fact questions

Metrics to watch

MetricWhy
Mean pairwise similarity in final top-kRedundancy actually present
Distinct doc_ids / parents in top-kCoverage
Multi-fact answer completeness (eval)The benefit
Single-fact accuracyThe risk