How much human review is needed: scale it to risk

A customer service lead used Claude this morning for three things: ideas for the team offsite, the weekly internal email and a reply to a customer threatening legal action over a delayed refund, which offers a specific compensation. She has a legal adviser available for half an hour. Her manager suggests that, “to be safe,” everything Claude produces should go through legal.

It sounds prudent, but it spends the half hour on a list of offsite ideas. And if it’s done the other way round, nothing reviewed, the customer reply goes out without anyone with legal judgment reading it.

Review is scaled

Human review isn’t yes or no. It ranges from a glance by whoever sends the text to a specialist’s sign-off. What decides how much is needed is the impact of the output being wrong:

• Does it leave the organization? A customer, a regulator, the public, investors.
• Does it commit to something? Money, a promise, a legal position.
• Does it affect someone’s health, safety or rights? Instructions for using a medical device, a decision about an employee.
• Is it hard to undo? A sent email, a printed leaflet, a published figure.
• Is there another check later? A text the team will rewrite, an A/B test with a small audience.

Situation	Review that fits
Internal, low-risk draft that someone will edit anyway	Quick check by whoever uses it
Text going out but with no new facts (shortening an approved release)	Check by whoever publishes it
New figures or claims for an audience that relies on them	Additional verification against the source
Health, safety, rights, legal or financial commitments	Expert review before use

Applying it to a queue of tasks

A communications team at a medical device company has five outputs to review and little time from the regulatory specialist and finance. What to send?

• A patient leaflet with cleaning and reuse instructions: affects safety. Regulatory specialist.
• A paragraph for investors with revenue growth that Claude calculated: a company result, published. Finance.
• Thirty email subject lines to be tested with a small audience: the test is the check.
• A draft FAQ that HR will rewrite: it already has another check.
• A short version of an approved press release, with no new claims: check by whoever publishes it.

Notice that what decides isn’t how polished it looks or how many people it reaches, but what happens if it’s wrong.

Human in the loop

For high-impact work, the idea is a human in the loop: a person with judgment reviews and decides before the output takes effect. It isn’t enough for someone to “take a look” if that person can’t judge the content; for a legal matter, the review has to come from someone who knows the subject.

Common traps

• Expert review for everything. It exhausts the specialists and delays what didn’t need it.
• No review because “Claude is usually right.” Being right most of the time doesn’t cover the case that matters.
• Prioritizing by reach (what reaches the most people) instead of impact. The internal email to forty people weighs less than the reply to a single customer threatening to sue.
• Confusing polish with accuracy. Coming out fast and well written doesn’t change the risk.
• Review by someone without the necessary knowledge.

On the exam

These scenarios give several tasks and a scarce review resource. Rank them by impact: what goes outside, commits the organization or touches health, money or rights goes to the expert; what is internal, low-risk or has a later check gets a light check. Be wary of options with “everything” or “nothing,” and of those that prioritize by number of readers or by how good the text looks.