Security → Data residency & retention
What it is
An AI system spreads data across more places than teams expect. Every one is a residency and retention question:
| Store | Contains | Typical retention question |
|---|---|---|
| Vector index | Embeddings (and often chunk text in payload) | Are embeddings personal data? (Increasingly treated as yes) |
| Chunk registry / document store | Full document text | Mirrors source retention |
| Trace store | Prompts, retrieved content, answers (observability & tracing) | Usually the longest-lived copy of sensitive content |
| Eval datasets | Sampled production queries and documents (golden dataset first) | Often forgotten in erasure |
| Memory store | User facts and past interactions (memory) | Subject-access and erasure requests |
| Caches | Semantic cache answers, retrieval caches (latency levers) | Short TTL, but must be invalidated on erasure |
| Provider-side | Prompts sent to the model API | Zero-retention endpoints; training opt-out |
| Backups | Everything above | The hardest erasure case |
Three commitments a customer contract typically requires:
- Residency: data is processed and stored in a named region and does not leave it.
- Zero retention / no training: the model provider does not store prompts beyond the request and does not train on them.
- Retention and erasure: data is deleted after a stated period, and on request within a stated SLA.
How to implement
- Inventory every store that touches customer data, with its region, retention period, encryption, and access controls. This document is the basis of every security review you'll face.
- Per-tenant policy configuration: region, allowed model providers/endpoints, retention periods, training opt-out, PII handling (PII).
- Route by tenant policy: regional model endpoints, regional vector clusters, regional object storage. Enforce in the client so a misconfiguration fails rather than silently routing cross-region.
- Use zero-retention endpoints where required, and record the provider's data-processing terms per model.
- Retention jobs per store with defined periods, plus an
erase(subject)operation covering all of them (how to forget). - Backup policy documented: usually "backups age out within N days" rather than surgical deletion — write it down and have legal approve it.
- Evidence: log erasure operations with verification results; produce a completion record per request (auditability).
Small example
from dataclasses import dataclass
@dataclass(frozen=True)
class TenantPolicy:
tenant_id: str
region: str # "eu-central-1" | "us-east-1" | "ap-southeast-2"
allowed_providers: frozenset[str] # {"anthropic-eu"} for EU-only tenants
zero_retention_required: bool
training_opt_out: bool
retention_days: dict # {"traces": 30, "memories": 365, "documents": None}
pii_action: str # "mask" | "block" | "allow"
self_hosted_only: bool = False
REGION_ENDPOINTS = {
"eu-central-1": {"llm": "https://api.eu.anthropic.com", "vector": "qdrant-eu.internal",
"objects": "s3://acme-eu-documents"},
"us-east-1": {"llm": "https://api.anthropic.com", "vector": "qdrant-us.internal",
"objects": "s3://acme-us-documents"},
}
def clients_for(policy: TenantPolicy):
"""Routing enforced structurally: a cross-region call cannot be constructed."""
ep = REGION_ENDPOINTS[policy.region]
if policy.self_hosted_only:
llm = SelfHostedClient(endpoint=INTERNAL_VLLM[policy.region]) # Self-hosted serving
else:
provider = next(iter(policy.allowed_providers))
assert provider.endswith(policy.region.split("-")[0]) or provider in REGION_PROVIDERS[policy.region], \
f"provider {provider} not permitted in {policy.region}"
llm = APIClient(base_url=ep["llm"],
headers={"anthropic-beta": "zero-retention"} if policy.zero_retention_required else {})
return {"llm": llm, "vector": connect(ep["vector"]), "objects": s3(ep["objects"])}
# ---- retention enforcement per store ----
RETENTION_JOBS = {
"traces": lambda days: trace_store.delete_older_than(days),
"memories": lambda days: memory_store.expire_older_than(days),
"eval_samples": lambda days: eval_store.delete_samples_older_than(days),
"caches": lambda days: cache.purge_older_than(min(days, 1)),
"documents": lambda days: doc_store.delete_soft_deleted_older_than(days), # Document versioning
}
def enforce_retention(policy: TenantPolicy):
results = {}
for store, days in policy.retention_days.items():
if days is None: # indefinite (e.g. contractual documents)
results[store] = "retained"; continue
results[store] = RETENTION_JOBS[store](days)
audit_log("retention_enforced", tenant=policy.tenant_id, results=results)
return results
# ---- subject erasure across EVERY store ----
ERASURE_TARGETS = ["documents", "chunks", "vectors", "parents", "page_images",
"memories", "traces", "eval_samples", "caches", "quarantine"]
def erase_subject(subject_id: str, tenant_id: str, reason="gdpr_request") -> dict:
op_id = str(uuid.uuid4())
audit_log("erasure_start", op_id=op_id, subject=subject_id, tenant=tenant_id, reason=reason)
outcomes = {t: ERASERS[t](subject_id, tenant_id) for t in ERASURE_TARGETS}
verification = {t: VERIFIERS[t](subject_id, tenant_id) for t in ERASURE_TARGETS} # prove it, don't assume
clean = all(verification.values())
audit_log("erasure_complete", op_id=op_id, outcomes=outcomes, verified=clean,
backup_note="Backups age out within 35 days per retention policy DP-07")
if not clean:
alert_erasure_incomplete(op_id, [t for t, ok in verification.items() if not ok])
return {"op_id": op_id, "verified": clean, "targets": outcomes}
def test_no_cross_region_calls():
"""CI gate: an EU tenant must be unable to reach a US endpoint."""
eu = TenantPolicy("t_eu", "eu-central-1", frozenset({"anthropic-eu"}), True, True,
{"traces": 30}, "mask")
c = clients_for(eu)
assert "eu" in c["llm"].base_url and "eu" in str(c["vector"])
Pros
- Unlocks regulated and EU/public-sector customers — often the difference between winning a deal and not.
- Bounded exposure: shorter retention means less data at risk in a breach.
- Zero-retention endpoints remove the largest third-party concern in most security reviews.
- Documented inventory and policy turns each security questionnaire from a research project into a lookup.
- Per-tenant policy allows one platform to serve varied compliance needs.
Cons
- Regional deployments multiply infrastructure cost and operational surface — separate indexes, separate model endpoints, separate storage per region.
- Model availability differs by region: the best model may not be available where a tenant requires processing.
- Short trace retention hurts debugging and eval-set growth (data flywheel) — a genuine trade-off.
- Erasure across derived data is hard: embeddings, summaries, caches, eval sets, and backups all hold copies (how to forget).
- Backups are usually handled by policy rather than surgical deletion, which some customers challenge.
When to use
Any system serving EU, UK, healthcare, financial, or public-sector customers — and any system whose contracts mention data processing, which is most B2B systems.
When NOT to use (full regionalisation)
- Single-region, single-jurisdiction internal tools: apply retention and zero-retention, skip multi-region infrastructure until a customer requires it.
Production at scale
The data inventory is the foundational artifact. Every store, its region, its retention, its encryption, its access controls, and its erasure procedure. Security reviews, DPAs, and incident response all start here, and teams that lack it discover their trace store has been retaining prompts indefinitely.
Trace stores are the sleeper risk. They contain queries, retrieved document content, and answers — often the richest concentration of sensitive data you hold — and they're typically configured with generous retention for debugging. Set an explicit period, sample rather than retain everything (observability & tracing), and include them in erasure.
Embeddings are increasingly treated as personal data, since they can be partially inverted. Assume they're in scope for residency and erasure; don't argue the point in a customer review.
Zero-retention and training opt-out per provider and model — record which endpoints and terms apply, and re-verify on provider changes. This is a common question and the answer must be precise, not "we think so".
Enforce routing structurally: the client construction should make a cross-region call impossible, and CI should test it. Configuration alone drifts.
Erasure needs verification, not just execution (how to forget): query every store afterwards and assert absence; log the verification. "We ran the delete" is not evidence.
Retention shortens the eval flywheel. If traces expire in 30 days, sample into a longer-lived, anonymised eval store within that window (online eval, data flywheel) — otherwise short retention silently starves your evaluation.
Sub-processor list: model providers, vector store vendors, observability tools, and reranker APIs are all sub-processors that must appear in your DPA. Keep the list current as tools change (model & supply-chain hygiene).
Prod checklist
- Complete data inventory: every store, region, retention, encryption, access, erasure procedure
- Per-tenant policy: region, providers, zero-retention, training opt-out, retention periods, PII action
- Routing enforced in client construction; cross-region calls impossible; CI test
- Zero-retention endpoints and training opt-out configured and documented per provider
- Retention jobs per store, running and monitored
-
erase_subjectcovering documents, chunks, vectors, parents, images, memories, traces, eval samples, caches, quarantine - Erasure verification and completion records with SLA tracking
- Backup policy documented and legally reviewed
- Trace retention explicitly set; anonymised eval sampling within the window
- Sub-processor list current and reflected in the DPA
Metrics to watch
| Metric | Why |
|---|---|
| Erasure request → verified completion time | Contractual/legal SLA |
| Erasure verification failures (must be 0) | Incomplete deletion |
| Cross-region calls detected (must be 0) | Residency compliance |
| Data older than its retention period, per store | Policy enforcement |
| Sub-processor list drift | DPA accuracy |
Related
- How to forget — memory erasure mechanics
- Observability — trace store governance
- PII, Auditability, Supply chain
- Environments — regional and per-environment separation
- Self-hosted serving — the residency answer when APIs can't comply