Structure extraction → Page numbers + bounding boxes (verifiable citations)

RAG Citations With Page Numbers and Bounding Boxes Explained

What it is

A citation that says "source: contract.pdf" makes the user open a 90-page PDF and search. A citation that says "contract.pdf, page 47" is better. A citation that opens page 47 with the exact clause highlighted is what makes people trust and use the system — and what lets reviewers verify answers at scale.

Bounding boxes (x0, y0, x1, y1 in page coordinates) come from the parser: PDF text extraction gives per-span boxes; OCR gives per-line boxes; layout models give per-region boxes. Carry them through chunking (a chunk maps to a list of (page, bbox) for its constituent elements) and into the citation payload. The UI renders the page image or PDF with overlays.

How to implement

  1. Capture at parse: every element gets page (1-based), bbox (in a declared coordinate system: PDF points, origin bottom-left; or pixel coordinates with the page size), and page_size.
  2. Normalise coordinates to a single convention (e.g. PDF points, origin top-left, y down) at the common-schema boundary; store coord_system version.
  3. Chunk-level provenance: a chunk stores prov: [{page, bbox, element_id, char_range}] — one entry per element or per line it contains; not just the first page.
  4. Citation payload at generation time: chunk_id → doc_id, version, title, page(s), bbox(es), source_url, char offsets. Require the model to cite chunk IDs (citations); resolve to page/bbox server-side; never let the model invent page numbers.
  5. Render: page image (stored at ingest, scanned page) or PDF.js with highlight overlays; deep link …/doc/{id}?v={version}&page=47&hl=<bbox>.
  6. Printed page labels ≠ physical index: store both (page_index = 47, page_label = "A-3" from the PDF's page labels or the footer). Cite the label users see.
  7. Validate: bbox within page bounds; non-empty; text elements have non-degenerate boxes (production per-page heuristics coordinates check).

Small example

import fitz

def extract_with_boxes(path):
    doc = fitz.open(path)
    elements = []
    for pno, page in enumerate(doc, start=1):
        W, H = page.rect.width, page.rect.height
        label = page.get_label() or str(pno)                     # printed label if the PDF has page labels
        for b in page.get_text("blocks"):                        # (x0,y0,x1,y1,text,block_no,block_type)
            x0, y0, x1, y1, text, _, btype = b
            if btype != 0 or not text.strip():
                continue
            elements.append({"page_index": pno, "page_label": label, "page_size": [W, H],
                             "bbox": [x0, y0, x1, y1], "coord_system": "pdf-pt-topleft-v1", "text": text})
    return elements

def chunk_provenance(chunk_elements):
    """Merge element boxes per page into a few highlight rectangles."""
    by_page = {}
    for e in chunk_elements:
        by_page.setdefault(e["page_index"], []).append(e["bbox"])
    prov = []
    for p, boxes in by_page.items():
        x0 = min(b[0] for b in boxes); y0 = min(b[1] for b in boxes)
        x1 = max(b[2] for b in boxes); y1 = max(b[3] for b in boxes)
        prov.append({"page_index": p, "bbox_union": [x0, y0, x1, y1], "bboxes": boxes})
    return prov

# citation resolution at answer time
def resolve_citation(chunk_id, chunk_store):
    c = chunk_store.get(chunk_id)
    return {"title": c["title"], "page": c["prov"][0]["page_label"] if "page_label" in c["prov"][0] else c["prov"][0]["page_index"],
            "url": f"/doc/{c['doc_id']}?v={c['version']}&page={c['prov'][0]['page_index']}&hl={c['prov'][0]['bbox_union']}"}

Pros

  • Verifiable answers → user trust, faster review, fewer escalations.
  • Debuggability: a bad answer traces to the exact region the model read.
  • Highlight UX is the feature users mention first in enterprise RAG.
  • Enables human review workflows (bounded escalation with visible failure) and annotation for golden sets (test-time validation against a golden sample).
  • Regulated domains often require source-region provenance.

Cons

  • Coordinate chaos: PDF points vs pixels, origin bottom-left vs top-left, rotated pages, cropped pages, scaled renders. Bugs here show as highlights in the wrong place.
  • Storage: boxes per line for 100M chunks is real space; needs a document store, not the vector payload (metadata on every chunk).
  • Non-paginated sources (HTML, DOCX, Markdown) have no pages; need an alternative anchor (element XPath, char offsets, heading anchor).
  • Re-rendering requires storing page images or serving the PDF; both have cost and access-control implications.

When to use

  • Any user-facing Q&A over documents — the moment users must verify.
  • Regulated or high-stakes domains (legal, finance, healthcare, compliance).
  • Any system with human review of parser output or model answers.

When NOT to use

  • Sources with no visual page (chat logs, DB rows): cite by record ID/URL and char offsets instead; keep the same provenance schema shape with page=null.

Production at scale

One coordinate system, declared and versioned. Convert at the parser adapter boundary. Store page_size with every box so renders at any scale are correct. Write a unit test per parser adapter that round-trips a known box.

Rotation and cropping. Use the page's effective transform (PyMuPDF page.rotation_matrix, CropBox vs MediaBox). OCR'd pages: boxes are in the rasterised image's pixel space — store the DPI/scale used and convert.

Provenance storage. Vector payload gets page_index (filterable, small). Document store gets the full prov list keyed by chunk_id. The citation resolver fetches from the document store; cache hot chunks.

Access control on rendering. The highlight endpoint serves document pages; it must apply the same ACL check as retrieval (authentication & authorization). A leak here is worse than a leak in retrieval because it serves the full page.

Page images vs PDF. Store page images (WebP, ~100–200 KB/page) for scanned and mixed documents; serve PDF via PDF.js for digital ones. Images are simpler to secure and render consistently; PDFs are smaller for text-heavy documents. Many teams store both for the first N pages and render on demand beyond.

Non-paginated sources. Anchor by element path (HTML XPath / CSS selector, DOCX paragraph index) plus char offsets, and deep-link to the source URL with a text fragment (#:~:text=...). Keep page=null, anchor_type="xpath".

Model-invented citations. Never trust page numbers in model output. Require chunk IDs; validate every cited ID exists in the retrieved set; resolve pages server-side (citations). Unverifiable citations are dropped and counted as a metric.

Bounding-box drift after re-parse. A parser upgrade moves boxes slightly; citations stored in past answers (audit logs, saved reports) point at the old version. Citations must include version; keep old versions renderable for the retention window (document versioning).

Evals. Citation accuracy: does the highlighted region contain the supporting text? Automated check: string-match the cited chunk text against the text within the bbox on the rendered page (test-time validation against a golden sample); human sample for the rest.

Prod checklist

  • page_index, page_label, bbox, page_size, coord_system on every element
  • Coordinate normalisation at adapter boundary; round-trip unit tests per parser
  • Chunk prov list (all pages/boxes), stored in document store; page_index in vector payload
  • Citation resolver server-side from chunk IDs; model never emits page numbers
  • Highlight endpoint with the same ACL check as retrieval
  • Page images stored for scanned/mixed docs; PDF.js for digital
  • Non-paginated anchor scheme (XPath/offsets + text fragment)
  • Citations carry version; old versions renderable for retention window
  • Citation-accuracy eval (bbox text contains cited text)

Metrics to watch

MetricWhy
Citation accuracy (highlight contains supporting text)Trust
Unverifiable-citation rate (model cited a non-retrieved ID)Generation discipline
Bbox validation failures per parserCoordinate bugs
Highlight endpoint p95 and ACL denialsUX and security