Communicating value and limitations to stakeholders

A customer care team ran a six-week pilot: agents used Claude to draft replies to written complaints and corrected every draft before sending it. Drafting time fell by about 30%, and one draft in ten needed a substantial correction, almost always in billing details. The director has already told the board that Claude will “fully automate replies next quarter” and plans to shrink the team.

The team lead who ran the pilot has the numbers needed to stop a bad decision. The question is how to present them.

What the decision-makers need to know

Stakeholders will not use Claude every day; they will decide budget, staffing and risk based on what you tell them. A good briefing has four parts:

1. Measured results: what was measured, against what, and how much it changed. “Review time per contract went from about 90 minutes to 50 across 40 contracts” is worth more than “it saves a lot of time.”
2. What Claude does and does not do: it writes the first draft, flags clauses, groups comments. It does not approve, send or decide.
3. The limitations the pilot showed, with numbers: “it missed two non-standard clauses, which the lawyers caught in their review”; “one draft in ten had billing errors.”
4. Where human review stays and why: the errors found are precisely the reason.

The errors do not weaken the proposal: they make it credible. A chief financial officer who asks for “a number we can trust” distrusts a pilot with no failures.

Managing expectations

Managing expectations is part of the job. If someone has already promised full automation, the briefing has to show with data why that does not follow from the pilot: the 30% saving happened with an agent correcting every draft. Without that correction, one customer in ten would receive a wrong billing detail.

Adapt the emphasis to each stakeholder, without changing the data:

Stakeholder	What worries them	What to highlight
Finance	That the number is real	Pilot measurement, sample size, cost of review time
Legal or compliance	That errors do not get out	What slipped through, who caught it, which control remains
Department leadership	Team capacity	Hours freed up and what they will be used for

What does not build trust

• Figures from vendor brochures about “AI in general.”
• Projecting one team’s saving to the whole company without testing it in other teams.
• Promising that human review will disappear “once it is up and running.”
• Hiding the numbers for fear they will be misread: without data, the decision gets made anyway, less informed.
• Declaring the pilot a failure because there were errors, when the saving was real and the errors were caught.

The typical traps

• Backing the full-automation plan because the saving “proves it is ready.”
• Recommending “more testing” without giving any numbers.
• Scrapping the pilot over a 10% correction rate.
• Using generic benchmarks instead of your own results.

On the exam

If the scenario asks what to communicate or what makes a proposal credible, look for the option that combines the measured value with the observed limitations and explains where human review stays. In “select two” items, the pilot results (including the errors) and the description of the controls usually go together. Rule out promises of full automation, untested extrapolations, marketing figures and silence about the numbers.