Prompt engineering → Decomposition, negative examples, and prefilling
What it is
Prompt decomposition. One prompt that does five things (classify, extract, decide, draft, format) fails unpredictably. Splitting it into a pipeline of narrow prompts — each with its own eval, tier (model selection & tradeoffs), and validation — makes each step reliable and debuggable. Different from query decomposition (decomposition), which splits a user question for retrieval; this splits the task.
Negative examples / "do not". Models follow "do X" well and "don't do Y" reasonably — much better when you show the wrong output labelled as wrong next to the right one. Best used for a small number of recurring, specific failures ("do not include the disclaimer"; "do not convert currencies"); long lists of don'ts dilute.
Prefilling. You write the first tokens of the assistant's reply — { to force JSON, <answer> to skip preamble, Subject: to force an email format, a table header to force a table. The model continues from there. Removes chatter ("Sure! Here's…"), enforces structure, and can steer persona/format more reliably than instructions alone. (Supported by Anthropic's Messages API; check other providers — some don't allow it, and some models with extended thinking disallow it.)
How to implement
Decomposition
- List the sub-tasks in the monolithic prompt. For each: input, output schema, quality bar, failure mode.
- Make each a call site with its own prompt, tier, validator, and eval (prompt versioning & evaluation, prompt evaluation).
- Chain deterministically in code; pass structured outputs (structured outputs & tool use) between steps; fail fast on validation errors.
- Parallelise independent steps.
Negative examples
- From triage (feedback capture & triage), pick the top 3–5 recurring mistakes.
- For each, add a
<bad_example>with the wrong output and a one-line reason, next to a<good_example>. - Keep the "do not" list short; verify each item still matters on the golden set every quarter — remove the ones that no longer fire.
Prefilling
- Decide the required opening (
{,<answer>,|,Subject:). - Add an assistant message with that text as the last message; the response continues it.
- Re-attach the prefill to the parsed output (the model's text starts after the prefill).
- Combine with schema validation; prefill helps, it doesn't guarantee.
Small example
# --- Decomposition: one monolith → three narrow calls ---
def handle_ticket(ticket):
label = llm("intent_classifier", CLASSIFY_SYS, [{"role": "user", "content": ticket}]).text.strip() # fast tier
fields = extract_json("ticket_extract", EXTRACT_SYS, ticket) # fast tier, schema-validated
if label == "BILLING" and fields.get("invoice_id"):
facts = billing_api.lookup(fields["invoice_id"]) # deterministic step
else:
facts = retrieve(ticket, caller) # RAG
return llm("reply_draft", DRAFT_SYS, [{"role": "user", "content": f"<label>{label}</label><facts>{facts}</facts><ticket>{ticket}</ticket>"}]) # balanced tier
# --- Negative example inside the draft prompt ---
DRAFT_SYS = """Write a reply to the customer using only <facts>.
<good_example>
<facts>Invoice 8812 was paid on 2024-03-02.</facts>
<output>Hi — invoice 8812 shows as paid on 2 March 2024. Let us know if you need the receipt.</output>
</good_example>
<bad_example reason="invents a refund timeline not in facts">
<output>Hi — invoice 8812 is paid and your refund will arrive in 3-5 days.</output>
</bad_example>
Do not promise timelines or amounts that are not in <facts>."""
# --- Prefilling: force JSON with no preamble ---
def extract_json(call_site, system, text):
r = client.messages.create(model=TIER[call_site], max_tokens=400, temperature=0, system=system,
messages=[{"role": "user", "content": f"<ticket>{text}</ticket>"},
{"role": "assistant", "content": "{"}]) # prefill
raw = "{" + r.content[0].text # re-attach
return validate_or_retry(raw, TicketFields) # Structured outputs & tool use
Pros
- Decomposition: reliability, per-step evals and tiers, cheaper (small steps on fast models), parallelism, clear failure attribution.
- Negative examples: cheap, targeted fix for recurring specific errors.
- Prefilling: near-elimination of preamble and format drift; fewer retries; steers tone/format without long instructions.
Cons
- Decomposition: more calls → more latency (mitigate with parallelism), more prompts to maintain, error propagation between steps if outputs aren't validated.
- Negative examples: can backfire (the model imitates the bad example) if not clearly labelled; lists of don'ts bloat and dilute.
- Prefilling: provider-dependent; incompatible with some reasoning modes; you must re-attach the prefix; can over-constrain (forces JSON even when the right response is a refusal — design the schema to allow it).
When to use
- Decompose when one prompt has >2 distinct jobs, or when failures are hard to attribute.
- Negative examples for the top recurring mistakes from triage.
- Prefill for any structured output, and for skipping chatter in pipelines.
When NOT to use
- Don't decompose a task the model does reliably in one call — extra hops cost latency.
- Don't accumulate "do not" rules for one-off issues; fix the root cause (retrieval, schema) instead.
- Don't prefill when the model must be free to refuse or ask a clarifying question — or make the schema include those options.
Production at scale
Decomposed pipelines are workflows — represent them as a graph with typed edges, retries per step, and tracing (observability & tracing); a step failure should be visible with its input. This is the "prefer a fixed workflow with LLM steps" advice from agents in miniature (planning styles).
Latency budget per pipeline: sum of critical-path steps; parallelise independent ones; use fast tiers for narrow steps. Stream the final step only.
Prefill + schema + retry is the standard structured-output stack when native schema enforcement isn't available (structured outputs & tool use). With native structured outputs, prefill becomes unnecessary for JSON but still useful for skipping preamble in free text.
Negative-example hygiene: each has an owner and a linked eval case; review quarterly; remove when the eval passes without it.
Model upgrades: prefill behaviour and negative-instruction adherence differ across models; include both in the upgrade eval (versioning & rollout).
Prod checklist
- Monolithic prompts split into call sites with schemas, tiers, validators, evals
- Pipeline as a traced workflow; independent steps parallel; latency budget tracked
- Negative examples limited to top recurring failures; labelled clearly; linked to eval cases; reviewed quarterly
- Prefill used for structured outputs; prefix re-attached; schema allows refusal/clarification
- Provider compatibility of prefill verified per model/tier
Metrics to watch
| Metric | Why |
|---|---|
| Per-step accuracy and failure attribution | Decomposition value |
| Recurrence rate of each negative-example failure | Whether the rule still matters |
| Preamble/format-violation rate with vs without prefill | Prefill value |
| Pipeline p95 latency | Cost of extra hops |
Related
- Structured outputs — validation + retry
- Few-shot — positive examples
- Query decomposition — the retrieval-side cousin
- Planning styles — scripted workflows with LLM steps
- Feedback triage — source of negative examples