Structure extraction → Page numbers + bounding boxes (verifiable citations)
What it is
A citation that says "source: contract.pdf" makes the user open a 90-page PDF and search. A citation that says "contract.pdf, page 47" is better. A citation that opens page 47 with the exact clause highlighted is what makes people trust and use the system — and what lets reviewers verify answers at scale.
Bounding boxes (x0, y0, x1, y1 in page coordinates) come from the parser: PDF text extraction gives per-span boxes; OCR gives per-line boxes; layout models give per-region boxes. Carry them through chunking (a chunk maps to a list of (page, bbox) for its constituent elements) and into the citation payload. The UI renders the page image or PDF with overlays.
How to implement
- Capture at parse: every element gets
page(1-based),bbox(in a declared coordinate system: PDF points, origin bottom-left; or pixel coordinates with the page size), andpage_size. - Normalise coordinates to a single convention (e.g. PDF points, origin top-left, y down) at the common-schema boundary; store
coord_systemversion. - Chunk-level provenance: a chunk stores
prov: [{page, bbox, element_id, char_range}]— one entry per element or per line it contains; not just the first page. - Citation payload at generation time: chunk_id →
doc_id, version, title, page(s), bbox(es), source_url, char offsets. Require the model to cite chunk IDs (citations); resolve to page/bbox server-side; never let the model invent page numbers. - Render: page image (stored at ingest, scanned page) or PDF.js with highlight overlays; deep link
…/doc/{id}?v={version}&page=47&hl=<bbox>. - Printed page labels ≠ physical index: store both (
page_index= 47,page_label= "A-3" from the PDF's page labels or the footer). Cite the label users see. - Validate: bbox within page bounds; non-empty; text elements have non-degenerate boxes (production per-page heuristics
coordinatescheck).
Small example
import fitz
def extract_with_boxes(path):
doc = fitz.open(path)
elements = []
for pno, page in enumerate(doc, start=1):
W, H = page.rect.width, page.rect.height
label = page.get_label() or str(pno) # printed label if the PDF has page labels
for b in page.get_text("blocks"): # (x0,y0,x1,y1,text,block_no,block_type)
x0, y0, x1, y1, text, _, btype = b
if btype != 0 or not text.strip():
continue
elements.append({"page_index": pno, "page_label": label, "page_size": [W, H],
"bbox": [x0, y0, x1, y1], "coord_system": "pdf-pt-topleft-v1", "text": text})
return elements
def chunk_provenance(chunk_elements):
"""Merge element boxes per page into a few highlight rectangles."""
by_page = {}
for e in chunk_elements:
by_page.setdefault(e["page_index"], []).append(e["bbox"])
prov = []
for p, boxes in by_page.items():
x0 = min(b[0] for b in boxes); y0 = min(b[1] for b in boxes)
x1 = max(b[2] for b in boxes); y1 = max(b[3] for b in boxes)
prov.append({"page_index": p, "bbox_union": [x0, y0, x1, y1], "bboxes": boxes})
return prov
# citation resolution at answer time
def resolve_citation(chunk_id, chunk_store):
c = chunk_store.get(chunk_id)
return {"title": c["title"], "page": c["prov"][0]["page_label"] if "page_label" in c["prov"][0] else c["prov"][0]["page_index"],
"url": f"/doc/{c['doc_id']}?v={c['version']}&page={c['prov'][0]['page_index']}&hl={c['prov'][0]['bbox_union']}"}
Pros
- Verifiable answers → user trust, faster review, fewer escalations.
- Debuggability: a bad answer traces to the exact region the model read.
- Highlight UX is the feature users mention first in enterprise RAG.
- Enables human review workflows (bounded escalation with visible failure) and annotation for golden sets (test-time validation against a golden sample).
- Regulated domains often require source-region provenance.
Cons
- Coordinate chaos: PDF points vs pixels, origin bottom-left vs top-left, rotated pages, cropped pages, scaled renders. Bugs here show as highlights in the wrong place.
- Storage: boxes per line for 100M chunks is real space; needs a document store, not the vector payload (metadata on every chunk).
- Non-paginated sources (HTML, DOCX, Markdown) have no pages; need an alternative anchor (element XPath, char offsets, heading anchor).
- Re-rendering requires storing page images or serving the PDF; both have cost and access-control implications.
When to use
- Any user-facing Q&A over documents — the moment users must verify.
- Regulated or high-stakes domains (legal, finance, healthcare, compliance).
- Any system with human review of parser output or model answers.
When NOT to use
- Sources with no visual page (chat logs, DB rows): cite by record ID/URL and char offsets instead; keep the same provenance schema shape with
page=null.
Production at scale
One coordinate system, declared and versioned. Convert at the parser adapter boundary. Store page_size with every box so renders at any scale are correct. Write a unit test per parser adapter that round-trips a known box.
Rotation and cropping. Use the page's effective transform (PyMuPDF page.rotation_matrix, CropBox vs MediaBox). OCR'd pages: boxes are in the rasterised image's pixel space — store the DPI/scale used and convert.
Provenance storage. Vector payload gets page_index (filterable, small). Document store gets the full prov list keyed by chunk_id. The citation resolver fetches from the document store; cache hot chunks.
Access control on rendering. The highlight endpoint serves document pages; it must apply the same ACL check as retrieval (authentication & authorization). A leak here is worse than a leak in retrieval because it serves the full page.
Page images vs PDF. Store page images (WebP, ~100–200 KB/page) for scanned and mixed documents; serve PDF via PDF.js for digital ones. Images are simpler to secure and render consistently; PDFs are smaller for text-heavy documents. Many teams store both for the first N pages and render on demand beyond.
Non-paginated sources. Anchor by element path (HTML XPath / CSS selector, DOCX paragraph index) plus char offsets, and deep-link to the source URL with a text fragment (#:~:text=...). Keep page=null, anchor_type="xpath".
Model-invented citations. Never trust page numbers in model output. Require chunk IDs; validate every cited ID exists in the retrieved set; resolve pages server-side (citations). Unverifiable citations are dropped and counted as a metric.
Bounding-box drift after re-parse. A parser upgrade moves boxes slightly; citations stored in past answers (audit logs, saved reports) point at the old version. Citations must include version; keep old versions renderable for the retention window (document versioning).
Evals. Citation accuracy: does the highlighted region contain the supporting text? Automated check: string-match the cited chunk text against the text within the bbox on the rendered page (test-time validation against a golden sample); human sample for the rest.
Prod checklist
-
page_index,page_label,bbox,page_size,coord_systemon every element - Coordinate normalisation at adapter boundary; round-trip unit tests per parser
- Chunk
provlist (all pages/boxes), stored in document store;page_indexin vector payload - Citation resolver server-side from chunk IDs; model never emits page numbers
- Highlight endpoint with the same ACL check as retrieval
- Page images stored for scanned/mixed docs; PDF.js for digital
- Non-paginated anchor scheme (XPath/offsets + text fragment)
- Citations carry
version; old versions renderable for retention window - Citation-accuracy eval (bbox text contains cited text)
Metrics to watch
| Metric | Why |
|---|---|
| Citation accuracy (highlight contains supporting text) | Trust |
| Unverifiable-citation rate (model cited a non-retrieved ID) | Generation discipline |
| Bbox validation failures per parser | Coordinate bugs |
| Highlight endpoint p95 and ACL denials | UX and security |
Related
- Scanned page OCR — page images
- Reading order — element IDs
- Chunk metadata — where provenance lives
- Citations — generation-side contract
- AuthN/AuthZ — rendering must be filtered too