claudio.cool
CCAO-FLessons

CCAO-F · Troubleshooting and Optimisation

Turning feedback into requirements

⏱ 3 min readDraft

The marketing team at an industrial supplies distributor uses Claude to draft the descriptions for the online catalog. Marketing is happy: they come out fast and read well. The e-commerce team, which publishes them, sends many back. They say required specifications such as load rating and materials are missing, and that the upbeat consumer tone does not suit the purchasing managers who use the catalog.

Who is right? The one who uses the output.

Who judges the output

The first lesson of the domain already established that the most useful feedback comes from whoever receives the result. This one is about how to turn it into concrete changes.

The fact that whoever writes the prompt is happy says nothing about whether the output works. Marketing measures “it reads well”; e-commerce measures “it can be published.” The second one is what matters, because if it is not published, the work was wasted.

From complaint to requirement

“They don’t work for us” cannot go into a prompt. It has to be translated into requirements Claude can meet:

  1. Ask for accepted and rejected examples. Ten descriptions e-commerce published as they were and ten it sent back, with the reason.
  2. Find the pattern. What do the accepted ones have that the rejected ones lack? You can ask Claude to help compare them, but e-commerce validates the conclusion.
  3. Write it as a concrete instruction. Not “more technical,” but: “always include load rating, material and dimensions; if one is missing from the product sheet, mark it as MISSING instead of inventing it; informative tone for professional buyers, no exclamation marks.”
  4. Turn it into acceptance criteria: the list used to decide whether a description works. It is the same list you will use to measure the change.
What the complaint says Requirement that goes into the prompt
“Too long” “Maximum 80 words, main use first”
“Doesn’t work for buyers” “Audience: purchasing managers; informative tone”
“Data is missing” “Always load, material and size; if missing, flag it”
“It makes things up” “Use only the data from the sheet I give you”

Test small, with whoever complained

Before redoing 150 priority descriptions, test the adjusted prompt on a test batch of ten or fifteen, and review it with e-commerce. If they approve, you move ahead; if not, you adjust again. Change one thing at a time so you know what worked.

Keep measuring

Feedback is not an event, it is a loop. After rolling out:

  • keep count of how many outputs are accepted without changes;
  • every new rejection, with its reason, is a case to add to the test set;
  • if the rejection rate climbs again, check what changed (the products, the audience, the prompt).

The typical traps

  • Leaving the prompt as it is because whoever writes it is happy.
  • Changing the prompt, the model and the template at once to “fix everything in one round.”
  • Asking Claude to grade itself from 1 to 10 and sending only the 9s: it is the same tool judging itself.
  • Translating the complaint into an adjective (“more professional”) instead of a requirement.
  • Applying the change to the whole catalog without testing it on a small batch.

On the exam

If the scenario shows the recipients rejecting the output, look for the option that collects examples of what they accept and reject and turns them into prompt requirements, and the one that tests the change on a small batch with them before scaling. In “select two” items, they usually go together. Rule out the author’s satisfaction, Claude’s self-assessment and changes made all at once.

Quick question

A practice exam question on this domain. It counts towards your progress.

The accounts payable team at a construction firm uses a shared prompt in a Claude Project to sort incoming supplier emails into urgent, normal and no action. For the first months it worked well and the team trusted its labels.

Since then, different team members have added lines to the instructions whenever something went wrong. The prompt now includes "Always mark as urgent if an invoice is mentioned" near the top and, further down, "Only mark as urgent if payment is due within 7 days". Another line says "Treat reminders as no action".

This month, identical-looking emails get different labels from one day to the next, and a few overdue payment reminders were marked as no action.

One colleague says the model must have been changed and wants to try a different one; another proposes rewriting the whole prompt from scratch over the weekend and using it from Monday.

What should the team do first?

Practise this domain in a practice exam →