claudio.cool
CCAR-FLessons

CCAR-F · Context Management

When to escalate to a human and how to resolve ambiguity

⏱ 3 min readDraft

The support agent has to resolve most cases on first contact, but not all of them. If it escalates too much, it is useless; if it escalates too little, it makes decisions that are not its to make. The key is to escalate on criteria, not on impressions.

The triggers that work

The guide names three escalation triggers:

  1. The customer asks for a human. If they ask explicitly, you escalate right away, without investigating first or trying to resolve, even if the case is within the agent’s authority.
  2. The policy has a gap or calls for an exception (a policy gap). If the customer asks to match a competitor’s price and the policy only covers price adjustments on the company’s own site, the policy says nothing about that request. The agent does not invent a rule: it escalates.
  3. The agent cannot make progress. After trying, no progress is possible with its tools.

What is not a trigger: the case being complex. A complex case that the policy covers gets resolved.

Anger is not the same as asking for a human

A furious customer whose package arrived smashed, with a case the policy clearly covers, does not need a human: they need a resolution. The agent acknowledges the frustration, apologises and offers the replacement. If the customer insists on talking to someone, then it escalates.

Situation What the agent does
“I want to talk to a supervisor” Escalates right away
Angry customer, case covered by policy Acknowledges, offers to resolve; escalates only if they reiterate
Request the policy does not address Escalates
$250 refund with authority up to $100 Resolves what it can and escalates the rest with a summary
One message with a return and a question about a product recall Handles the return, escalates only the other part

Why sentiment and confidence do not work

Sentiment-based escalation measures tone, not complexity: it escalates angry customers with simple cases and lets polite customers with out-of-policy requests through. The confidence the model reports about itself does not work either: it is poorly calibrated, and the agent is wrong with complete certainty. If confidence-based routing lets errors through, replace it with explicit escalation criteria and measure escalations against labelled outcomes.

Those criteria go in the system prompt with examples of when to escalate and when to resolve autonomously, boundary cases included.

Multiple matches

get_customer returns two accounts with the same name. The agent does not pick the “most likely” one (the most recent, the one with more orders): it requests an additional identifier, such as the order number or email, before touching any account. The same if the customer says “cancel my order” and has three open: it lists them and asks which one. Acting on a guess can refund or cancel something for the wrong person.

Unknown state

If process_refund timed out twice and you do not know whether the refund went through, do not retry blindly (you might pay twice) and do not confirm it (you would be stating something you do not know). Tell the customer the status is unknown and escalate so someone checks it.

When it escalates, the handoff carries a structured summary (facts, what has been done, the open question); “customer unhappy, escalating” helps nobody.

The typical traps

  • Investigating or trying to resolve when the customer has already asked for a human.
  • Escalating because of anger, or escalating everything complex even when it is covered.
  • Escalating based on the model’s confidence.
  • Picking an account or an order by heuristic.

On the exam

Read exactly what the customer said. “I want a person” escalates right away; “this is a disaster” does not. Then look at the policy: if it covers the case, resolve it; if it is silent, escalate. And if there is more than one match, the answer asks for another piece of data.

Quick question

A practice exam question on this domain. It counts towards your progress.

A developer at a home-appliance retailer is building a multi-turn support agent on the Messages API to help customers troubleshoot washing machines and dishwashers. To keep costs down, the code sends each request with only the latest three messages of the conversation, plus the system prompt. The prototype was tested internally for two weeks and the system prompt has been rewritten twice.

Now that real users are trying it, they report that the agent forgets details they gave earlier in the conversation, such as the model number or the error code shown on the display, and asks for them again a few turns later. The developer had assumed the API would keep the earlier part of the conversation on its side.

What is the cause, and what is the fix?

Practise this domain in a practice exam →