Security → PII (detect, mask, reversible mapping)
What it is
Personal data flows through an AI system at three points, each needing a different decision:
| Point | What flows | Options |
|---|---|---|
| Ingest | Documents containing PII | Index as-is (inside your boundary), redact, or tokenise; tag chunks with sensitivity |
| Request | User's query, attachments | Mask before third-party call; block; allow |
| Response | Generated answer, possibly containing PII from context | Restore masked values for the entitled user; check for leakage to others (output guardrails) |
The key architectural insight: mask at the trust boundary, not at the front door. Retrieval needs real names and identifiers to work — BM25 matches "Jane Okafor", dedupe compares real text, and filters use real entity values. If you mask at ingress, "Jane's contract" becomes "PERSON_1's contract" and retrieval degrades badly. The right pattern for most systems:
query (real) → retrieval inside your boundary (real values, ACL-filtered)
│
mask ONLY what crosses to a third-party model
│
model sees <PERSON_1>, <EMAIL_2>, <ACCOUNT_3>
│
restore in the answer, for the entitled caller only
Reversible tokenisation preserves referential consistency: the same person is <PERSON_1> everywhere in one request, so the model can still reason about relationships.
How to implement
- Classify sensitivity at ingest (metadata on every chunk): tag chunks with detected PII categories so downstream policy can act, and so high-sensitivity content can be routed to a self-hosted model (self-hosted serving).
- Detect with a real NER pipeline (Presidio, spaCy, cloud DLP) plus deterministic patterns (national IDs, card numbers, IBANs) with checksum validation to cut false positives.
- Consistent pseudonymisation within a request: same entity → same placeholder, so coreference survives.
- Request-scoped mapping only — never persist the mapping; it's the re-identification key.
- Restore only for the caller who supplied the values and only for entities they're entitled to see (authentication & authorization).
- Per-tenant policy (data residency & retention): mask, block, or allow, per category.
- Never log raw PII — log entity types, counts, and offsets; guardrail logs are themselves a PII risk.
Small example
from presidio_analyzer import AnalyzerEngine
from presidio_anonymizer import AnonymizerEngine
import re, hashlib
analyzer = AnalyzerEngine()
IBAN = re.compile(r"\b[A-Z]{2}\d{2}[A-Z0-9]{11,30}\b")
def luhn_ok(num: str) -> bool:
d = [int(c) for c in re.sub(r"\D", "", num)][::-1]
return sum(x if i % 2 == 0 else sum(divmod(x * 2, 10)) for i, x in enumerate(d)) % 10 == 0
class PIIMasker:
"""Consistent, reversible, request-scoped tokenisation."""
def __init__(self):
self.counters, self.mapping, self.reverse = {}, {}, {}
def _token(self, entity_type: str, value: str) -> str:
if value in self.reverse:
return self.reverse[value] # same entity → same placeholder
self.counters[entity_type] = self.counters.get(entity_type, 0) + 1
tok = f"<{entity_type}_{self.counters[entity_type]}>"
self.mapping[tok], self.reverse[value] = value, tok
return tok
def mask(self, text: str, entities: list[str], threshold=0.6) -> str:
findings = [f for f in analyzer.analyze(text=text, language="en",
entities=entities, score_threshold=threshold)]
for m in IBAN.finditer(text): # deterministic patterns supplement NER
findings.append(Finding("IBAN", m.start(), m.end(), 1.0))
findings.sort(key=lambda f: -f.start) # replace right-to-left to keep offsets valid
for f in findings:
value = text[f.start:f.end]
if f.entity_type == "CREDIT_CARD" and not luhn_ok(value):
continue # checksum reduces false positives
text = text[:f.start] + self._token(f.entity_type, value) + text[f.end:]
return text
def restore(self, text: str) -> str:
for tok, value in self.mapping.items():
text = text.replace(tok, value)
return text
def summary(self) -> dict:
return {t.split("_")[0].strip("<"): 0 for t in self.mapping} # TYPES only — never log values
# ---- mask at the trust boundary, not at ingress ----
def answer_with_pii_boundary(query: str, caller, policy):
chunks = retrieve(query, caller) # real values: retrieval quality preserved
if policy.pii_action == "allow" or policy.self_hosted_only:
return generate_grounded(query, chunks) # stays inside the boundary; no masking needed
masker = PIIMasker()
masked_query = masker.mask(query, policy.pii_entities)
masked_chunks = [c | {"text": masker.mask(c["text"], policy.pii_entities)} for c in chunks]
log_pii(caller.tenant_id, entity_types=list(masker.summary().keys())) # types only
out = generate_grounded(masked_query, masked_chunks) # third party sees only placeholders
out["answer"] = masker.restore(out["answer"]) # entitled caller sees real values
return out # masker discarded with the request
# ---- ingest-side classification for routing and policy ----
def classify_chunk_sensitivity(text: str) -> dict:
findings = analyzer.analyze(text=text, language="en", entities=SENSITIVE_ENTITIES, score_threshold=0.7)
cats = sorted({f.entity_type for f in findings})
level = "high" if {"MEDICAL_LICENSE", "US_SSN", "IBAN_CODE"} & set(cats) else "medium" if cats else "none"
return {"pii_categories": cats, "sensitivity": level} # stored on the chunk (metadata on every chunk)
Pros
- Enables third-party model use in environments that would otherwise require self-hosting.
- Reversible tokenisation preserves answer usefulness — the user still sees real names.
- Consistent placeholders preserve reasoning: the model can still track who did what.
- Boundary masking preserves retrieval quality, unlike ingress masking.
- Sensitivity tags enable per-chunk routing (self-hosted model for high-sensitivity content).
Cons
- Detection is imperfect: NER misses unusual name formats and over-flags common words; recall and precision both matter, and both are language-dependent.
- Masking degrades some answers — the model reasons less well about
<PERSON_1>than about a named individual in some tasks. - The mapping is a re-identification key; persisting it anywhere is a serious risk.
- Latency: NER over long contexts adds tens to hundreds of milliseconds.
- Multilingual coverage is weak outside major languages.
- Structured PII in tables and forms often evades text-oriented NER (tables).
When to use
- Any flow sending customer data to a third-party model where policy or contract requires it (data residency & retention).
- High-sensitivity categories (health, financial, government ID) regardless of tenant, as a default posture.
- Logging and eval pipelines — always scrub before storing samples (online eval).
When NOT to use
- Fully self-hosted, in-boundary processing where no third party sees the data — masking then costs quality for no privacy gain.
- At ingest, if it degrades retrieval — classify and tag at ingest, mask at the boundary.
- As a substitute for access control: masking is not authorisation (authentication & authorization).
Production at scale
Mask at the boundary, tag at ingest. Ingest-time classification (which categories, what sensitivity) drives routing and policy; runtime masking happens only where data crosses to a third party. This preserves retrieval quality and applies protection exactly where it's needed.
Never persist the mapping. It lives in request memory, is used to restore the answer, and is discarded. Persisting it — in a cache, a trace, or a session store — recreates the exposure you were preventing.
Traces and eval samples must be scrubbed before storage (observability & tracing, online eval). This is a commonly missed path: masking the model call but logging the raw prompt.
Tune thresholds per language and domain and measure both directions: missed PII (a leak) and false positives (quality damage). Checksum validation for structured identifiers is a cheap precision win.
Route by sensitivity: chunks tagged high-sensitivity can be handled by a self-hosted model (self-hosted serving) rather than masked, preserving quality for the cases where masking hurts most.
Structured PII needs separate handling: names and IDs in tables and form fields (complex layout, tables) often bypass text NER. Apply column-level rules where you have structure.
Right of access and erasure apply to everything derived from personal data — embeddings, summaries, memories, traces (data residency & retention, how to forget).
Measure quality with masking on. Run the golden set through the masked path; if quality drops materially, that's a real cost to weigh against the policy, and it argues for self-hosting on sensitive tenants.
Prod checklist
- Sensitivity classification at ingest, stored on chunks; used for routing and policy
- NER + deterministic patterns with checksum validation; thresholds tuned per language
- Masking applied at the third-party boundary, not at ingress
- Consistent, request-scoped, reversible tokenisation; mapping never persisted
- Restoration only for the entitled caller; output leakage check for others (output guardrails)
- Per-tenant policy (mask / block / allow) per category
- Traces, logs, and eval samples scrubbed before storage
- Never log raw values — types, counts, offsets only
- High-sensitivity content routed to in-boundary models where available
- Golden-set quality measured with masking enabled
Metrics to watch
| Metric | Why |
|---|---|
| PII detection rate by category and language | Coverage |
| False-positive rate (review sample) | Quality damage |
| Quality delta with masking on vs off | The real cost of the policy |
| Raw PII found in traces/logs (must be 0) | Scrubbing gaps |
| Masking latency p95 | Request-path cost |
Related
- Input guardrails, Output guardrails — where masking runs
- Residency & retention — the policy source
- Chunk metadata — sensitivity tags
- Self-hosted serving — the alternative to masking
- Memory security — PII at the memory write gate