Security → Model & supply-chain hygiene

AI Supply Chain Security Explained: Dependencies Can Ship Prompts

What it is

AI systems have the usual software supply chain — packages, containers, base images — plus three novel surfaces:

SurfaceRiskWhy it's different
Model weightsBackdoored or trojaned open-weight models; pickle deserialisation RCE in .bin checkpointsWeights are opaque; you cannot read them for malice
Third-party tool servers (MCP)Tool descriptions are instructions your model follows; results enter your contextA malicious description is a prompt injection you installed deliberately (prompt-injection defence)
Prompt/agent packagesCommunity prompts, agent templates, and skill files containing hidden instructionsText executed by a model, not a compiler
Model provider changesUnpinned aliases changing behaviour; deprecations forcing rushed migrationsBehaviour changes without a deploy (versioning & rollout)
Data poisoningAdversarial content in ingested corporaPersistent, and reaches every future query (source ingestion, prompt-injection defence)

The rule that captures the novelty: anything that contributes text to a prompt is executable content. Tool descriptions, system-prompt fragments, retrieved documents, and MCP server metadata all get read and acted on by the model.

How to implement

  1. Pin everything: model IDs (never aliases), weight file hashes, serving-engine versions, package versions with lockfiles, container digests (not tags).
  2. Prefer safetensors over pickle-based checkpoints; verify SHA-256 against the publisher's published hash; scan for unexpected pickle imports if you must load them.
  3. Review third-party MCP servers before enabling: read every tool description, check what data it accesses and what network calls it makes, pin the version, and re-review on upgrade.
  4. Run untrusted tool servers with least privilege: separate process or container, no credentials, egress allow-list (sandboxing, excessive agency).
  5. Treat tool descriptions as prompt assets under version control and review (prompt versioning & evaluation, tool design); diff them on every dependency update.
  6. SBOM and vulnerability scanning for containers and packages, plus an AI-specific inventory: models, adapters, embedding models, rerankers, and their sources.
  7. Track deprecations and provider changes as scheduled work, with eval runs on every change (versioning & rollout).

Small example

import hashlib, json, subprocess

# ---- Pin and verify model artefacts ----
MODEL_MANIFEST = {
    "generation": {"provider": "anthropic", "model_id": "claude-sonnet-5",   # exact ID, never an alias
                   "deprecation_date": "2026-03-01", "terms": "zero-retention-eu"},
    "embedding":  {"source": "huggingface", "repo": "intfloat/multilingual-e5-large",
                   "revision": "ab10c1a7f42e2b0c",                            # commit, not "main"
                   "sha256": "9f2e...c41a", "format": "safetensors"},
    "reranker":   {"source": "huggingface", "repo": "BAAI/bge-reranker-v2-m3",
                   "revision": "5f8d2c9e1b34", "sha256": "3b7d...80fe", "format": "safetensors"},
}

def verify_model_artifact(path: str, expected_sha: str, fmt: str):
    if fmt != "safetensors":
        raise ValueError("refusing pickle-format weights: use safetensors")   # pickle load = RCE
    h = hashlib.sha256()
    with open(path, "rb") as f:
        for block in iter(lambda: f.read(1 << 20), b""):
            h.update(block)
    if h.hexdigest() != expected_sha:
        raise SecurityError(f"weight hash mismatch for {path}")
    return True

# ---- MCP server review gate ----
APPROVED_MCP_SERVERS = {
    "acme-knowledge": {"version": "1.4.2", "reviewed": "2024-06-10", "reviewer": "sec-team",
                       "descriptions_hash": "b31f...9a2c", "egress": [], "credentials": "caller_delegated"},
}

INSTRUCTION_LIKE = re.compile(
    r"(ignore\s+(previous|all)|you\s+must\s+always|system\s*:|do\s+not\s+tell\s+the\s+user|"
    r"send\s+(this|the\s+\w+)\s+to\s+https?://)", re.I)

def vet_mcp_server(name: str, manifest: dict) -> dict:
    """Run before enabling ANY third-party tool server."""
    findings = []
    approved = APPROVED_MCP_SERVERS.get(name)
    if not approved:
        findings.append("server not in the approved list")
    elif manifest["version"] != approved["version"]:
        findings.append(f"version drift: {manifest['version']} != approved {approved['version']}")

    desc_blob = json.dumps([t["description"] for t in manifest["tools"]], sort_keys=True)
    desc_hash = hashlib.sha256(desc_blob.encode()).hexdigest()
    if approved and desc_hash != approved["descriptions_hash"]:
        findings.append("tool descriptions changed since review — re-review required")

    for tool in manifest["tools"]:                       # descriptions are text the model OBEYS
        if INSTRUCTION_LIKE.search(tool["description"]):
            findings.append(f"instruction-like content in description of {tool['name']}")
        if tool.get("input_schema", {}).get("properties", {}).keys() & {"tenant_id", "user_id", "acl_groups"}:
            findings.append(f"{tool['name']} accepts identity parameters — must come from session (authentication & authorization)")
    return {"approved": not findings, "findings": findings, "descriptions_hash": desc_hash}

def enable_mcp_server(name, manifest, caller_context):
    v = vet_mcp_server(name, manifest)
    if not v["approved"]:
        audit_log("mcp_server_rejected", server=name, findings=v["findings"])
        raise SecurityError(f"MCP server {name} rejected: {v['findings']}")
    return MCPClient(name, credentials=None,                       # no ambient credentials (excessive agency)
                     identity=caller_context,                      # delegated, per request
                     egress_allowlist=APPROVED_MCP_SERVERS[name]["egress"])

# ---- CI: dependency and description drift ----
def test_no_unpinned_models():
    for role, m in MODEL_MANIFEST.items():
        assert "latest" not in str(m.get("model_id", "")) and m.get("revision") != "main", \
            f"{role} is unpinned"

def test_deprecations_not_imminent(days=60):
    for role, m in MODEL_MANIFEST.items():
        if d := m.get("deprecation_date"):
            assert (date.fromisoformat(d) - date.today()).days > days, \
                f"{role} deprecates in <{days} days — migration must be scheduled (versioning & rollout)"

Pros

  • Prevents behaviour changes you didn't authorise — the most common real incident, from an unpinned model alias.
  • Description review catches installed injections before they reach production.
  • Hash verification and safetensors eliminate the weight-loading RCE class.
  • Deprecation tracking turns forced migrations into planned work (versioning & rollout).
  • Least-privilege tool servers bound the damage from a compromised dependency.

Cons

  • Review effort per dependency, and the MCP ecosystem moves fast enough that pinning means falling behind.
  • Pinning creates upgrade debt; security patches still need applying.
  • Description review doesn't scale to dozens of servers without tooling.
  • Open-weight provenance is hard to verify beyond publisher hashes — you cannot inspect weights for backdoors.
  • Vetting slows adoption of genuinely useful third-party tools.

When to use

Any production system. The MCP-review gate specifically applies the moment you enable a tool server you did not write.

When NOT to use (heavy vetting)

  • First-party tools in your own codebase go through normal code review; the extra gate is for third-party and community-sourced servers, prompts, and models.

Production at scale

Tool descriptions are prompt assets. Store them in version control, hash them, and diff on every dependency update. A dependency upgrade that changes a tool description is a prompt change and must go through eval and review (prompt versioning & evaluation, tool design).

No ambient credentials in tool servers. A third-party server should receive delegated, per-request identity and hold nothing of its own (excessive agency, authentication & authorization). If it needs a credential, it needs a review that treats it as a system with production access.

Egress allow-lists for tool servers and sandboxes (sandboxing). The exfiltration path is the risk; a server that cannot reach the internet cannot send your data anywhere.

Reject identity parameters in tool schemas. A tool that accepts tenant_id or user_id as an argument lets the model choose whose data to access. Identity comes from the authenticated session, never from arguments — check this in vetting.

Maintain an AI-specific inventory alongside your SBOM: models, adapters, embedding models, rerankers, tool servers, and prompt packages, each with source, version, hash, licence, and review date. Security reviews will ask for it.

Licence review for open-weight models — some prohibit commercial use or impose conditions. This is a real blocker discovered late by many teams.

Provider change monitoring: subscribe to model deprecation notices, track dates as tickets, and run the full eval suite on any provider-announced change (versioning & rollout).

Data poisoning is a supply-chain risk too: content ingested from external sources can carry injections that persist in your index (prompt-injection defence). Screen at ingest, track the source, and be able to purge everything from a compromised connector.

Prod checklist

  • All model IDs pinned exactly; no aliases, no main revisions
  • Weight hashes verified; safetensors required; pickle checkpoints refused
  • Third-party MCP servers reviewed before enabling: descriptions read, egress checked, version pinned
  • Tool-description hashes tracked; drift triggers re-review
  • Tool schemas rejected if they accept identity parameters
  • Tool servers run with delegated identity, no ambient credentials, egress allow-lists
  • SBOM plus AI-specific inventory (models, adapters, rerankers, servers, prompts) with licences
  • Deprecation dates tracked; CI fails when a sunset is imminent
  • Container digests pinned; images scanned and rebuilt on a schedule
  • Ingest-source tracking enabling purge of a compromised connector

Metrics to watch

MetricWhy
Unpinned dependencies/models (target 0)Uncontrolled behaviour change
Tool-description drift eventsInstalled-injection risk
Days to nearest model deprecationMigration runway
Third-party servers enabled without review (must be 0)Gate integrity
Vulnerability findings in images/packagesOrdinary supply chain