OCR / digitization → Scanned page (PaddleOCR)

Scanned Page OCR for RAG Explained: Validate, Never Trust

What it is

Optical character recognition detects text regions on an image and recognises the characters in each. Modern OCR (PaddleOCR PP-OCRv4/v5, Tesseract 5, Surya, cloud OCR APIs) is a two-stage pipeline: detection (where are the text lines) → recognition (what do they say), optionally with layout analysis (complex layout) and table recognition on top. Output per line: text, bounding box, confidence.

PaddleOCR is the usual open-source pick because it is fast, multilingual (80+ languages), GPU-accelerated, and ships a layout model (PP-Structure) for the complex-layout route.

Scanned input quality varies wildly: 300 dpi office scans are easy; faxes, phone photos, skewed pages, handwriting, stamps over text, and carbon copies are not. Preprocessing matters as much as the model.

How to implement

  1. Rasterise the page at 200–300 dpi (PyMuPDF page.get_pixmap(dpi=300)). Lower loses small text; higher wastes GPU.
  2. Preprocess: deskew (Hough/angle detection), binarise or contrast-normalise, denoise lightly, crop borders. Most OCR engines have built-in angle classification (use_angle_cls=True) — turn it on.
  3. Run detection + recognition with a GPU-backed engine; batch pages.
  4. Assemble lines into blocks using bbox geometry (line spacing, x-alignment) — or use the layout model (complex layout) if the page isn't simple single-column.
  5. Keep confidence per line; compute page-level stats (mean, % lines < 0.7).
  6. Validate (production per-page heuristics): min text, garbage ratio, dictionary-word ratio, confidence. Fail → retry ladder (bounded escalation with visible failure): a second engine or a VLM.
  7. Emit common schema elements with page, bbox, confidence, parser=paddleocr@version.
  8. Store the page image (or a reference) for citation highlighting and human review.

Small example

import fitz, numpy as np
from paddleocr import PaddleOCR

ocr = PaddleOCR(use_angle_cls=True, lang="en", use_gpu=True, show_log=False)   # load once per worker

def rasterise(page: fitz.Page, dpi: int = 300) -> np.ndarray:
    pix = page.get_pixmap(dpi=dpi, colorspace=fitz.csRGB)
    return np.frombuffer(pix.samples, dtype=np.uint8).reshape(pix.height, pix.width, 3)

def ocr_page(page: fitz.Page, page_no: int) -> dict:
    img = rasterise(page)
    result = ocr.ocr(img, cls=True)[0] or []
    lines = [{"bbox": box, "text": txt, "conf": float(conf)} for box, (txt, conf) in result]
    lines.sort(key=lambda l: (round(l["bbox"][0][1] / 12), l["bbox"][0][0]))    # rough reading order: rows then x
    confs = [l["conf"] for l in lines]
    return {
        "page": page_no,
        "elements": [{"type": "line", "text": l["text"], "bbox": l["bbox"], "confidence": l["conf"],
                      "page": page_no, "parser": "paddleocr@2.8"} for l in lines],
        "stats": {"n_lines": len(lines), "mean_conf": float(np.mean(confs)) if confs else 0.0,
                  "low_conf_ratio": float(np.mean([c < 0.7 for c in confs])) if confs else 1.0},
    }

Pros

  • Only way to get text from scans, faxes, photos.
  • Confidence scores give a built-in quality signal for validation and routing.
  • Bounding boxes enable citation highlighting on the page image.
  • PaddleOCR is fast on GPU (tens of pages/sec on a modern card for simple pages) and multilingual.

Cons

  • Error rate is never zero: 1–3% CER on clean scans, 10–30% on bad ones. Errors in numbers and names are the ones that matter and the ones OCR gets wrong most.
  • GPU cost and latency: 100× slower than digital extraction.
  • No structure by itself: lines, not paragraphs/headings/tables. Needs layout analysis for anything beyond simple pages.
  • Preprocessing sensitivity: skew, low contrast, and compression artefacts tank accuracy.
  • Handwriting, stamps, and overlapping text are largely unsolved for general OCR; need specialised models or a VLM.

When to use

When NOT to use

  • Pages with a good text layer — OCR is slower and less accurate than reading the layer.
  • Handwritten forms, heavy stamps, or degraded historical documents where a VLM or specialised model is required — go straight to the VLM tier of the ladder.
  • When the page is really a chart or diagram (charts, images, diagrams).

Production at scale

GPU worker pool. OCR engine loaded once per worker; workers pull pages (not documents) from a queue so long scans are spread across the pool. Batch pages per inference call (PaddleOCR supports batched recognition). Autoscale on queue depth. Track GPU utilisation; underutilised GPUs usually mean rasterisation (CPU) is the bottleneck — move rasterisation to a CPU pre-stage that writes images to object storage.

DPI policy. 300 dpi for text-heavy pages; 200 for large-print; cap image dimensions (e.g. 4,000 px long side) to bound memory. Adaptive: if low-conf ratio > threshold at 200 dpi, retry at 300 before escalating engines.

Numbers and IDs are the risk. OCR confuses 0/O, 1/l/I, 5/S, 8/B. For domains where numbers matter (finance, contracts, medical), add post-OCR validation: regex-check expected formats (dates, amounts, policy numbers), cross-check totals in tables, and flag low-confidence numeric tokens for the retry ladder or human review.

Confidence calibration. Engine confidence is not a probability; calibrate on your golden sample (test-time validation against a golden sample) to pick the threshold that actually separates good from bad pages for your corpus. Store both raw and calibrated.

Language routing. Detect language per page (from the first pass or from document metadata) and select the right recognition model; a Latin model on Arabic or CJK pages returns garbage with high confidence.

Retry ladder integration. Fail criteria from validation → second engine (e.g. Tesseract or a cloud OCR) → VLM → human queue. Bound it; each rung is more expensive. Log the rung that succeeded; if the VLM is winning most escalations for a source, consider routing that source straight there.

Store the images. Page images (WebP/JPEG, moderate quality) in object storage, keyed by doc_id/version/page. Needed for citation highlighting, human review, and re-OCR when you upgrade the engine without re-fetching the source.

Cost per page is the metric leadership asks about. Track GPU-seconds and $ per page by source; compare engine versions on both accuracy and cost before upgrading.

Security. Scanned documents often contain the most sensitive content (signed contracts, IDs). OCR workers handle raw images; keep them in the ingest network segment with no egress, and never send images to a third-party OCR API without residency/retention checks (data residency & retention).

Prod checklist

  • Rasterise at a controlled DPI with size caps; deskew/angle-cls on
  • Engine loaded once per GPU worker; page-level queue; batched inference
  • Per-line confidence kept; page stats computed; calibrated threshold
  • Language detection → correct recognition model
  • Numeric/ID post-validation for domains where numbers matter
  • Validation → retry ladder with logged rung
  • Page images stored for citations, review, and re-OCR
  • GPU-seconds and $/page by source tracked
  • No third-party OCR without residency/retention approval

Metrics to watch

MetricWhy
Pages/sec per GPU; GPU utilisationCapacity, bottleneck location
Mean confidence, low-conf ratio per sourceInput quality drift
CER/WER on golden sample per engine versionUpgrade gate
Escalation rate to rung 2/3Engine adequacy
$/pageUnit economics