Prompt engineering → Decomposition, negative examples, and prefilling

3 Prompt Engineering Techniques: Decomposition, Negative Examples, Prefill

What it is

Prompt decomposition. One prompt that does five things (classify, extract, decide, draft, format) fails unpredictably. Splitting it into a pipeline of narrow prompts — each with its own eval, tier (model selection & tradeoffs), and validation — makes each step reliable and debuggable. Different from query decomposition (decomposition), which splits a user question for retrieval; this splits the task.

Negative examples / "do not". Models follow "do X" well and "don't do Y" reasonably — much better when you show the wrong output labelled as wrong next to the right one. Best used for a small number of recurring, specific failures ("do not include the disclaimer"; "do not convert currencies"); long lists of don'ts dilute.

Prefilling. You write the first tokens of the assistant's reply — { to force JSON, <answer> to skip preamble, Subject: to force an email format, a table header to force a table. The model continues from there. Removes chatter ("Sure! Here's…"), enforces structure, and can steer persona/format more reliably than instructions alone. (Supported by Anthropic's Messages API; check other providers — some don't allow it, and some models with extended thinking disallow it.)

How to implement

Decomposition

  1. List the sub-tasks in the monolithic prompt. For each: input, output schema, quality bar, failure mode.
  2. Make each a call site with its own prompt, tier, validator, and eval (prompt versioning & evaluation, prompt evaluation).
  3. Chain deterministically in code; pass structured outputs (structured outputs & tool use) between steps; fail fast on validation errors.
  4. Parallelise independent steps.

Negative examples

  1. From triage (feedback capture & triage), pick the top 3–5 recurring mistakes.
  2. For each, add a <bad_example> with the wrong output and a one-line reason, next to a <good_example>.
  3. Keep the "do not" list short; verify each item still matters on the golden set every quarter — remove the ones that no longer fire.

Prefilling

  1. Decide the required opening ({, <answer>, |, Subject:).
  2. Add an assistant message with that text as the last message; the response continues it.
  3. Re-attach the prefill to the parsed output (the model's text starts after the prefill).
  4. Combine with schema validation; prefill helps, it doesn't guarantee.

Small example

# --- Decomposition: one monolith → three narrow calls ---
def handle_ticket(ticket):
    label = llm("intent_classifier", CLASSIFY_SYS, [{"role": "user", "content": ticket}]).text.strip()       # fast tier
    fields = extract_json("ticket_extract", EXTRACT_SYS, ticket)                                              # fast tier, schema-validated
    if label == "BILLING" and fields.get("invoice_id"):
        facts = billing_api.lookup(fields["invoice_id"])                                                       # deterministic step
    else:
        facts = retrieve(ticket, caller)                                                                       # RAG
    return llm("reply_draft", DRAFT_SYS, [{"role": "user", "content": f"<label>{label}</label><facts>{facts}</facts><ticket>{ticket}</ticket>"}])  # balanced tier

# --- Negative example inside the draft prompt ---
DRAFT_SYS = """Write a reply to the customer using only <facts>.
<good_example>
<facts>Invoice 8812 was paid on 2024-03-02.</facts>
<output>Hi — invoice 8812 shows as paid on 2 March 2024. Let us know if you need the receipt.</output>
</good_example>
<bad_example reason="invents a refund timeline not in facts">
<output>Hi — invoice 8812 is paid and your refund will arrive in 3-5 days.</output>
</bad_example>
Do not promise timelines or amounts that are not in <facts>."""

# --- Prefilling: force JSON with no preamble ---
def extract_json(call_site, system, text):
    r = client.messages.create(model=TIER[call_site], max_tokens=400, temperature=0, system=system,
        messages=[{"role": "user", "content": f"<ticket>{text}</ticket>"},
                  {"role": "assistant", "content": "{"}])                       # prefill
    raw = "{" + r.content[0].text                                                # re-attach
    return validate_or_retry(raw, TicketFields)                                  # Structured outputs & tool use

Pros

  • Decomposition: reliability, per-step evals and tiers, cheaper (small steps on fast models), parallelism, clear failure attribution.
  • Negative examples: cheap, targeted fix for recurring specific errors.
  • Prefilling: near-elimination of preamble and format drift; fewer retries; steers tone/format without long instructions.

Cons

  • Decomposition: more calls → more latency (mitigate with parallelism), more prompts to maintain, error propagation between steps if outputs aren't validated.
  • Negative examples: can backfire (the model imitates the bad example) if not clearly labelled; lists of don'ts bloat and dilute.
  • Prefilling: provider-dependent; incompatible with some reasoning modes; you must re-attach the prefix; can over-constrain (forces JSON even when the right response is a refusal — design the schema to allow it).

When to use

  • Decompose when one prompt has >2 distinct jobs, or when failures are hard to attribute.
  • Negative examples for the top recurring mistakes from triage.
  • Prefill for any structured output, and for skipping chatter in pipelines.

When NOT to use

  • Don't decompose a task the model does reliably in one call — extra hops cost latency.
  • Don't accumulate "do not" rules for one-off issues; fix the root cause (retrieval, schema) instead.
  • Don't prefill when the model must be free to refuse or ask a clarifying question — or make the schema include those options.

Production at scale

Decomposed pipelines are workflows — represent them as a graph with typed edges, retries per step, and tracing (observability & tracing); a step failure should be visible with its input. This is the "prefer a fixed workflow with LLM steps" advice from agents in miniature (planning styles).

Latency budget per pipeline: sum of critical-path steps; parallelise independent ones; use fast tiers for narrow steps. Stream the final step only.

Prefill + schema + retry is the standard structured-output stack when native schema enforcement isn't available (structured outputs & tool use). With native structured outputs, prefill becomes unnecessary for JSON but still useful for skipping preamble in free text.

Negative-example hygiene: each has an owner and a linked eval case; review quarterly; remove when the eval passes without it.

Model upgrades: prefill behaviour and negative-instruction adherence differ across models; include both in the upgrade eval (versioning & rollout).

Prod checklist

  • Monolithic prompts split into call sites with schemas, tiers, validators, evals
  • Pipeline as a traced workflow; independent steps parallel; latency budget tracked
  • Negative examples limited to top recurring failures; labelled clearly; linked to eval cases; reviewed quarterly
  • Prefill used for structured outputs; prefix re-attached; schema allows refusal/clarification
  • Provider compatibility of prefill verified per model/tier

Metrics to watch

MetricWhy
Per-step accuracy and failure attributionDecomposition value
Recurrence rate of each negative-example failureWhether the rule still matters
Preamble/format-violation rate with vs without prefillPrefill value
Pipeline p95 latencyCost of extra hops