Input, processing, output and feedback, what goes in code and what goes to the model

A pipeline takes every attachment from a shared inbox, asks Claude to extract supplier, amount and due date, and posts it to the ERP. Within a week duplicate invoices and nonsense records appear, because the inbox also receives delivery notes and newsletters. The team adds “ignore duplicates and anything that is not an invoice” to the prompt. The duplicates continue: the model never saw the previous emails. The problem wasn’t the prompt; it was that the design had a single stage.

The four stages

An end-to-end architecture has four parts, and the exam expects you to spot which one is missing:

Stage	What it solves	Examples
Input	That the model receives what it should, clean	Deduplicate, classify and discard, normalize formats, remove sensitive data
Processing	The work that needs language or judgment	Extract, summarize, classify, draft
Output	That what comes out is usable and safe	Schema validation, business rules, routing to human review
Feedback	Knowing what happened with each output	What was accepted, what was corrected, what was discarded

What goes in code, not in the prompt

The rule: if it can be solved with a deterministic check, it goes in code, in the input or output stage. An instruction in the prompt is a suggestion; code is always enforced.

• Duplicates: a file hash or the invoice number against what is already recorded. The model doesn’t have that history; no model does.
• What doesn’t belong: an upstream classifier (it can be a small model) that separates invoices from delivery notes before extraction, with its own error metric.
• Format: ask for structured output and apply schema validation before loading anything. If it fails, retry or send it to a review queue, never straight into the system.
• Business rules: out-of-range amounts, impossible dates or totals that don’t add up are checked in code.

This also decides where to put people. Sending every record to manual approval with 2,500 attachments a week eliminates the benefit; sending only what fails a validation or falls below a threshold puts review where the risk is.

No feedback, no improvement

The feedback loop is the most forgotten stage. An assistant that suggests contract redlines can run for months without anyone knowing which suggestions lawyers accept, because their edits stay in the document management system. Then prompts get changed based on anecdotes from the partner who complains the most, and nobody can say whether a change made things better or worse.

To close the loop:

1. Capture the reaction to each output: accepted, edited (and how), discarded.
2. Store it with the prompt and model version that produced it.
3. Use it to feed the evaluation set and to prioritize what to fix (by clause type, by category).
4. Compare every change against that data before shipping it.

What doesn’t help is jumping to a larger model, retrieving more context or adding another call that “improves” the output without that data: it is optimizing blind.

The usual traps

• Asking the model to detect duplicates or filter junk that can only be detected with data it doesn’t see.
• Strengthening the prompt with more examples after the instruction already failed by design.
• Loading into the target system without schema validation “because the model almost always answers correctly”.
• Human review of everything to compensate for an unfiltered input.
• Designing input, processing and output, and leaving feedback for “a second phase”.
• Changing the model or adding a rewrite step when nobody knows which outputs fail.

In the exam

Read the scenario looking for the missing stage. If the problem appears because the model receives what it shouldn’t (duplicates, unrelated documents), the answer is an input stage with checks in code. If outputs reach another system malformed, it’s validation at the output. If the team “doesn’t know what fails” or improves based on anecdotes, the feedback loop is missing. Be wary of options that solve everything inside the prompt or with a more capable model.