Zero-shot, few-shot or step-by-step reasoning, which to use depending on where it fails

A card issuer classifies disputes into five reasons with a prompt that only lists each code and a one-line definition. Overall accuracy is good, but almost all errors come from one pair: “goods not received” versus “goods not as described”, when the customer writes “the box arrived empty”. One analyst proposes 40 if-then rules; another, moving to the largest model right away. Neither looks at where it fails.

The technique comes from the error

The objective’s three techniques are not levels of sophistication: each fixes a different kind of failure.

What the evaluation shows	Technique that fits
The task is clear and the model solves it well with instructions	Zero-shot: don’t add anything
It confuses two neighboring categories, or format and tone come out uneven	Few-shot examples that show the boundary or the format
It fails on multi-step tasks: chained calculations, comparing several sources, applying rules in order	Chain-of-thought or more model thinking
It fails even though the prompt is good, and another model solves it with the same prompt	It’s not prompting: it’s model mismatch (lesson 1)

Notice that everything starts with an evaluation set that tells you where the errors are. “It sort of works” isn’t enough to choose.

Few-shot done right

Examples are among the most reliable ways to guide judgment, format and tone. For them to work:

• Few and varied: three to five is usually enough, covering the edge cases and with different wordings so the model doesn’t copy a pattern you didn’t intend.
• Representative: taken from real cases, not invented to be easy.
• Separated: inside <example> tags (several inside <examples>), so they aren’t confused with the instructions.

Against the 40-rule list: examples teach the judgment and generalize to the next wording; rules cover what’s already been seen, are fragile and make the prompt impossible to maintain.

Step-by-step reasoning: when it pays off

Asking the model to reason before answering improves multi-step tasks, at the cost of more output tokens and more latency. So it depends on the path:

• In a nightly process, brief reasoning before the label is an acceptable cost.
• In an interactive path with a latency target, use it only where the evaluation shows it helps.
• If the reasoning has to be processed separately from the answer, ask for it in its own tags (for example, <analysis> and then <answer>).

In current models, adaptive thinking decides how much to think based on the request’s complexity and the effort parameter. That parameter is the lever to raise or lower reasoning without rewriting the prompt.

What isn’t reasoning: asking the model for a confidence score and silently discarding whatever scores low. Self-reported scores aren’t a reliable measure, and discarding without notice loses real cases.

The usual traps

• An endless list of rules to cover every wording seen.
• Jumping to the largest model without first trying the cheaper adjustment.
• Examples that are all alike, which the model imitates to the letter.
• Examples loose in the instruction text, without tags.
• Step-by-step reasoning on every path, including the one with a latency target.
• Trusting the confidence the model says it has.

In the exam

Look in the scenario for the description of the error. If it concentrates on a boundary between two categories or on inconsistent format, the answer is few-shot with a few varied examples in tags. If it fails on multi-step tasks and there’s latency headroom, it’s step-by-step reasoning or more effort. If the scenario says the process runs overnight or latency doesn’t matter, that’s a hint to add reasoning. Discard rule lists and model changes as the first step.