OCR / digitization → Per-page classification and routing

RAG Per-Page Routing for Document Parsing Explained

What it is

Parsers are specialists. A native-PDF text extractor is fast and exact on born-digital pages and returns nothing on a scan. An OCR engine reads scans but is slower and introduces errors on pages that had a perfectly good text layer. A formula extractor understands LaTeX and mangles tables. A vision-language model describes charts and is far too expensive to run on 200 pages of plain text.

Per-page routing runs a cheap classifier on each page first and dispatches:

Page classSignalRoute
Digital textText layer present, high char density, fonts embeddedDocling / pdfplumber / PyMuPDF (digital document)
ScannedNo/garbage text layer, page is one large imagePaddleOCR (scanned page)
Complex layoutMulti-column, forms, boxes; layout model detects many regionsPaddleOCR layout / Docling layout model (complex layout)
Formula-heavyMath fonts, high symbol density, $/\ patternsMinerU (formula-heavy page)
Chart / figure / diagramImage regions dominate, few text tokensVLM captioning (charts, images, diagrams)
Blank / separatorNear-zero contentSkip

Fast path where possible, accurate path where necessary, expensive path only where nothing else works.

How to implement

  1. Open the PDF once (PyMuPDF) and iterate pages. For each page collect: text-layer char count, ratio of image area to page area, number of fonts, presence of math fonts / Unicode math ranges, number of layout blocks, page size.
  2. Rule-based classifier first. if chars > 200 and image_ratio < 0.3 → digital; if chars < 20 and image_ratio > 0.8 → scanned; if math_font or unicode_math_ratio > 0.05 → formula; if image blocks > text blocks → figure. Rules are transparent and debuggable; cover 90% of pages.
  3. Small ML classifier for the rest (optional): a tiny CNN or a layout-model pass on a downscaled page image, trained on a few hundred labelled pages from your corpus.
  4. Route each page to its parser; parsers return a common element schema (layout / structure-aware) with page, bbox, element_type, parser, confidence.
  5. Re-assemble pages into one document in order, preserving heading state across pages (a section starting on a digital page and continuing on a scanned one must not split).
  6. Validate per page (production per-page heuristics); failures go to the retry ladder (bounded escalation with visible failure).
  7. Record the route as metadata (parser, page_class) — you will need it when a specific page type turns out to be broken.

Small example

import fitz  # PyMuPDF
import unicodedata

MATH_RANGES = [(0x2200, 0x22FF), (0x1D400, 0x1D7FF), (0x2A00, 0x2AFF)]

def classify_page(page: fitz.Page) -> str:
    text = page.get_text("text")
    chars = len(text.strip())
    area = page.rect.width * page.rect.height
    img_area = sum(fitz.Rect(*im["bbox"]).get_area() for im in page.get_image_info()) if area else 0
    img_ratio = min(img_area / area, 1.0) if area else 0
    math_chars = sum(1 for ch in text if any(lo <= ord(ch) <= hi for lo, hi in MATH_RANGES))
    fonts = {f[3] for f in page.get_fonts()}
    blocks = page.get_text("blocks")
    if chars < 20 and img_ratio > 0.6:               return "scanned"
    if chars < 20:                                   return "blank"
    if math_chars / max(chars, 1) > 0.03 or any("Math" in f or "CM" in f for f in fonts):
        return "formula"
    if img_ratio > 0.5 and chars < 400:              return "figure"
    cols = len({round(b[0] / 50) for b in blocks if b[4].strip()})
    if cols >= 3 or len(blocks) > 40:                return "complex_layout"
    return "digital"

ROUTES = {"digital": parse_docling, "scanned": parse_paddle_ocr, "complex_layout": parse_paddle_layout,
          "formula": parse_mineru, "figure": parse_vlm_caption, "blank": lambda p: []}

def parse_pdf(path):
    doc = fitz.open(path)
    elements = []
    for i, page in enumerate(doc):
        cls = classify_page(page)
        els = ROUTES[cls](page)                        # each returns list of common-schema elements
        for e in els:
            e["page"], e["page_class"] = i + 1, cls
        elements.extend(els)
    return elements

Pros

  • Right tool per page: exact text where a text layer exists, OCR only where needed, VLM only on figures.
  • Cost: OCR/VLM spend drops by the fraction of pages that are digital — usually most of them.
  • Speed: digital extraction is 100× faster than OCR.
  • Quality: no OCR errors on pages that didn't need OCR; no missed content on scanned appendices.
  • Debuggability: a bad output traces to a page class and a parser.

Cons

  • Classifier errors route pages wrongly: a scanned page with a garbage OCR text layer from the source system looks "digital" and yields garbage. Need validation after parsing (production per-page heuristics) to catch it.
  • Page re-assembly must handle structure spanning pages; naive concatenation breaks headings and tables that cross page boundaries.
  • More moving parts: five parsers to version, monitor, and keep in a common schema.
  • Mixed pages (half text, half chart) need region-level routing, which is another level of complexity.

When to use

  • Any corpus with PDFs from the wild: scanned contracts with digital cover pages, reports with embedded charts, papers with equations.
  • Whenever OCR or VLM cost/latency is a constraint (it always is at scale).

When NOT to use

  • Homogeneous corpora: all born-digital HTML/Markdown/DOCX → one parser, no routing needed.
  • Tiny volumes where running the heavy parser on everything is affordable and simpler — but keep the route metadata anyway.

Production at scale

Two-pass design. Pass 1 (cheap, CPU): open, classify, extract digital pages. Pass 2 (expensive, GPU): OCR, layout, formula, VLM for the pages that need it. Queue pass-2 pages separately with page-level granularity so a 500-page scanned appendix doesn't block a 2-page digital memo. Different worker pools, different autoscaling.

Region-level routing inside a page. For mixed pages, run a layout model once (it's cheap relative to OCR) to get regions; route text regions to extraction/OCR and figure regions to the VLM. This is what Docling's pipeline and MinerU do internally — use their region outputs rather than rebuilding.

Garbage text layers. Some scanners embed low-quality OCR text. Detect with the decode-garbage heuristics (production per-page heuristics: replacement chars, dictionary-word ratio, char n-gram entropy); if the "digital" text fails, re-route to OCR. Make this a standard step, not an exception.

Cross-page structure. Maintain heading/section state across pages during re-assembly; detect table continuation (same column x-positions on consecutive pages, "continued" markers) and merge. Docling handles much of this when given the whole document; if you route pages to different parsers you must reassemble carefully or you lose it.

Cost accounting per page class. Log pages_by_class and cost_by_class per source. This tells you where money goes and whether a source's mix changed (a partner started sending scans instead of digital exports).

Classifier drift. New document templates shift feature distributions. Monitor the class mix per source; a sudden jump in "scanned" or "complex_layout" is either a real change or a classifier regression. Keep a labelled page set (a few hundred pages) and re-evaluate the classifier when you change any threshold.

Parser versions per route. Each parser has its own version; store parser=<name>@<version> on elements. A bad OCR release can be rolled back and only "scanned"-class pages re-parsed (change detection config-driven reprocessing, filtered by page class).

Timeouts and size caps. OCR on a 5,000-page scan can run for hours. Per-page timeouts, per-document page caps with an oversize path (sample pages, or split into parts), and cancellation.

Prod checklist

  • Cheap per-page feature extraction; rule-based classifier with documented thresholds
  • Garbage-text-layer detection → re-route to OCR
  • Region-level routing for mixed pages
  • Separate CPU (digital) and GPU (OCR/VLM) worker pools; page-level queues
  • Common element schema from every parser with page, bbox, parser@version, page_class
  • Cross-page re-assembly preserving heading/table continuity
  • Per-page timeouts; per-document page caps
  • Class-mix and cost-by-class dashboards per source
  • Labelled page set for classifier regression tests

Metrics to watch

MetricWhy
Pages by class per sourceMix drift, classifier regression
Cost and latency by classWhere the money goes
Post-parse validation failure rate by classMis-routing
Re-route rate (digital → OCR)Garbage text layers
Pass-2 queue depthGPU capacity