From "we want an agent" to the real problem

The operations director asks for “an agent that handles the claims”. IT wants to choose a model and a framework this month. Nobody knows how many claims come in by type or how long each one takes. If you start building with that, you will deliver something nobody can evaluate or defend. The architect’s first job is to turn the request into a problem.

Where the time or money goes today

Discovery is not a meeting to write down wishes: it is looking at the real process. The questions that organize everything:

• Which part of the process consumes the most hours or cost?
• Which decisions must a person make by law, by contract or because of risk?
• What volume is there per case type and how much risk does each carry?

Often the request is “an agent that decides” and the real bottleneck is reading, extracting or summarizing. If 70% of the time goes into transcribing amounts from attached reports and the final decision must be signed by a licensed professional, the solution is not to automate the decision: it is to extract the data in a structured format and flag the mismatches so the person decides faster. That is human-in-the-loop by design, not out of distrust.

Success metric and baseline, before the architecture

“Fewer tickets and happier employees” is not a success metric. A metric works if it can be measured, has a current value and someone agreed which value counts as success.

1. Pick the indicator the sponsor wants to move (handling time, cost per case, error rate).
2. Measure its baseline in the current process, before touching anything.
3. Map the cases by volume, effort and risk: high-volume, low-risk work is a candidate for automation; high-risk work (a complaint that may lead to an investigation) stays with people or with mandatory review.
4. Only then choose the pattern, model and tools.

Choosing the model or writing the system prompt before this step is starting at the end: those decisions come from the requirements, they don’t set them.

Tying the technology to business value

The guide talks about business value pillars: efficiency, productivity, transformation, cost and performance SLAs. They help you ask which pillar the project targets and which indicator represents it.

The sponsor asked for…	The indicator to measure
Analysts handling more cases	Cases per person and time per case
Lowering the department’s cost	Total cost per case, including corrections
Responses within the SLA	Latency by percentile in the flow that matters

The trap is optimizing what is easy to measure instead of what was asked for. A team that cuts a summary’s latency from 4 to 1.5 seconds did good technical work, but if the sponsor asked to reduce after-call work time and agents lose minutes fixing fields, another latency sprint moves nothing. Measure the business indicator and prioritize each change by its effect on it.

The usual traps

• Accepting “an agent” as a requirement instead of investigating the problem.
• Choosing the most capable model “so nothing has to be redone” before there are requirements.
• A multi-agent prototype of everything to impress, without knowing what is worth automating.
• Automating a decision that regulations say a person must make.
• Reporting technical improvements (latency, cost per request) without showing their effect on the sponsor’s indicator.

In the exam

Scenarios for this objective usually have a discovery paragraph with the key fact: where the time goes, which decision is mandatory for a person, or that nobody measures the indicator. Look for the option that attacks that bottleneck and leaves the regulated decision in human hands. In “what to do before choosing the architecture” questions, the good answers are defining the metric with a baseline and prioritizing cases by volume and risk; discard the ones that choose a model, write prompts or build prototypes first.