Structure extraction → Reading order
What it is
A PDF stores glyphs at coordinates; a scanned page stores pixels. Neither stores "read this paragraph, then that one". Parsers must infer reading order. Failure modes:
- Column interleaving: two-column text emitted line-by-line across columns.
- Sidebars/callouts injected mid-paragraph.
- Footnotes appearing where the marker is, or between paragraphs.
- Table cells interleaved with body text.
- Rotated text, headers/footers in the middle of content.
- Right-to-left scripts and vertical CJK.
Good parsers (Docling, MinerU, PP-Structure with recovery) use a layout model plus a reading-order model or XY-cut heuristic; still, verification is your job because errors are silent — the text is all there, just scrambled.
How to implement
- Use a layout-aware parser and its ordering output rather than raw extraction (digital document, complex layout).
- Column-aware fallback: cluster text blocks by x-position into columns; sort by column then y; then merge. XY-cut (recursively split the page at the widest horizontal/vertical gap) handles most layouts.
- Separate floating elements: tables, figures, captions, footnotes, sidebars, headers/footers are emitted as their own elements anchored to a position, not spliced into paragraphs.
- Sentence-continuity check: for consecutive text elements, if element A ends without terminal punctuation and element B starts lowercase (or vice versa in a suspicious way), that boundary is a candidate error. Score the page; high score → escalate (bounded escalation with visible failure).
- Hyphenation and line joins: join lines within a paragraph, de-hyphenate line-end hyphens (
inter-\nnational→international) using a dictionary check (keep genuine hyphens). - Language direction: detect RTL/vertical scripts and apply the correct ordering rule.
- Store
reading_indexon each element so the order is explicit and auditable.
Small example
def xy_cut(blocks, gap=18):
"""blocks: [(x0,y0,x1,y1,text)]. Recursive split on the widest gap; returns ordered blocks."""
if len(blocks) <= 1:
return blocks
for axis in (0, 1): # try vertical cut (columns) first, then horizontal
lo, hi = (0, 2) if axis == 0 else (1, 3)
s = sorted(blocks, key=lambda b: b[lo])
best, best_gap = None, gap
for i in range(1, len(s)):
g = s[i][lo] - max(b[hi] for b in s[:i])
if g > best_gap:
best, best_gap = i, g
if best:
return xy_cut(s[:best], gap) + xy_cut(s[best:], gap)
return sorted(blocks, key=lambda b: (b[1], b[0])) # no gap: top-to-bottom, left-to-right
def continuity_score(ordered_texts):
bad = 0
for a, b in zip(ordered_texts, ordered_texts[1:]):
a, b = a.rstrip(), b.lstrip()
if a and b and a[-1] not in ".!?:;\"')" and b[0].isupper():
bad += 1 # A trails off mid-sentence, B starts a new one
return bad / max(len(ordered_texts) - 1, 1)
import re
def dehyphenate(lines, vocab):
out = []
for i, ln in enumerate(lines):
if ln.endswith("-") and i + 1 < len(lines):
head, nxt = ln[:-1].split()[-1], lines[i + 1].split()[0]
if (head + nxt).lower() in vocab:
lines[i + 1] = lines[i + 1].replace(nxt, head + nxt, 1); ln = ln[: -len(head) - 1]
out.append(ln)
return " ".join(out)
Pros
- Sentences stay whole → embeddings mean something, and the LLM reads coherent text.
- Chunk boundaries fall on real paragraphs (recursive character splitting/04.1.3 depend on this).
- Tables and footnotes stop polluting body paragraphs.
- Explicit
reading_indexmakes citations and highlighting consistent with what the user reads.
Cons
- Heuristics fail on creative layouts (magazines, marketing, infographics).
- Layout model errors cascade: a missed column boundary scrambles the page.
- Verification is approximate: continuity checks miss errors on pages of short lines (lists, addresses).
- Extra processing per page; small but real.
When to use
Always — verification is cheap, and errors are silent. Especially for multi-column sources (papers, newsletters, statements, regulatory filings).
When NOT to use (the heavy machinery)
- Single-column born-digital documents and Office formats where order is explicit — just trust it, but keep the continuity check as a smoke test.
Production at scale
Trust the parser, verify the page. Compute the continuity score per page in the validation stage (production per-page heuristics). Threshold it per source (learn from the golden set). Failing pages → retry ladder with an alternative ordering strategy (XY-cut vs the parser's learned order) before escalating to a VLM.
Floating elements as first-class elements. The single most effective design decision: never splice a table, figure, footnote, or sidebar into paragraph text. Emit them as separate elements with an anchor_reading_index so the chunker can keep a table near its referencing paragraph without breaking the paragraph.
Footnotes. Detect by position (bottom region), small font, and leading marker; emit as footnote elements linked to their marker; append to the chunk of the referencing paragraph or index separately with the breadcrumb. Legal and financial corpora put material facts in footnotes.
Two-page spreads and rotated pages. Detect page rotation (PyMuPDF page.rotation, or from OCR angle classification) and landscape spreads; split spreads into two logical pages before ordering.
Language and script. RTL (Arabic, Hebrew) and vertical CJK need script-aware ordering; detect per page and switch rules. Mixed-script pages (Arabic with Latin numbers) are the hardest; test them explicitly.
Golden set for order. Keep ~50 pages with human-verified order; metric: pairwise order accuracy or Kendall tau over blocks (test-time validation against a golden sample). Run on parser upgrades.
Downstream tells. If chunk-level recall is poor on a multi-column source while single-column sources are fine, suspect ordering before anything else. Keep page_class and source in eval breakdowns.
Throughput. XY-cut and continuity checks are microseconds per page; de-hyphenation with a vocabulary lookup is cheap. No capacity concern; the concern is only doing it.
Prod checklist
- Layout-aware parser ordering used; XY-cut fallback available
- Tables/figures/footnotes/sidebars/headers as separate anchored elements
- Per-page continuity score in validation; per-source threshold
- De-hyphenation with dictionary check
- Rotation/spread handling; script-aware ordering for RTL/vertical
-
reading_indexstored on elements - Golden pages with verified order; order metric in the parsing eval
- Eval breakdown by page class/source to spot ordering regressions
Metrics to watch
| Metric | Why |
|---|---|
| Continuity score distribution per source | Ordering errors |
| Order accuracy (Kendall tau) on golden pages | Parser upgrades |
| Retry-ladder escalations for ordering | Parser fit per source |
| Chunk recall by page class | Downstream effect |
Related
- Digital document / complex layout — where ordering is produced
- Tables, Boilerplate — floating elements
- Production validation — where the check runs
- Recursive splitting — depends on coherent paragraphs