Few-shot examples for ambiguous cases and consistent format

You wrote detailed instructions and the output still differs on every run: dates in two formats, shifting severities, invented labels. When instructions alone do not achieve consistency, the most effective technique is to show few-shot examples: input and output pairs the model can imitate.

What they are for

• Consistent format: one example per input pattern (written date, numeric date, date with time), all with the output in the exact format you want. If a review finding has to include location, issue, severity and suggested fix, the examples show those four fields, always the same.
• Ambiguous cases: which tool to use for a request that could go to two, whether a branch coverage gap deserves a test, whether a review with a minor complaint is “positive” or “mixed”.
• Telling the acceptable from the problematic: code that looks like a bug but is an accepted team pattern, next to one that really is a problem. That cuts false positives.
• Documents with varied structures: inline citations or a bibliography at the end, methodology in its own section or embedded in the body, informal measurements (“two cups”, “a handful”). The examples show where to find the data in each structure and reduce both hallucinations and required fields that come back empty.

How to pick them

The guide talks about 2 to 4 targeted examples, not dozens. What teaches the most:

1. Boundary cases, with the why. Each example explains why that action was chosen and not the plausible alternative. With the reasoning, the model generalises the judgement to cases you did not anticipate, instead of looking for matches with the examples.
2. A balanced mix. Typical cases set the pattern; edge cases you have already seen fail mark the boundaries.
3. Every category represented. If both severity examples are “critical”, the model will say “critical” for almost everything. Rebalance so every level appears, with pairs that differ only on the point that separates two neighbouring levels.
4. The same mapping production needs, format details included. An example that does not look like the real case teaches a different task.

If the model invents labels for unusual requests, add unusual requests mapped to the closest allowed label and state that the output must be one from the list.

Measure instead of guessing

A classifier that gets the examples it has seen right can drop on new phrasings. Measure consistency on a held-out set the examples do not cover, and add examples only where it fails. If the prompt runs hundreds of thousands of times a day, test fewer examples on a labelled sample and keep the smallest set that holds the result.

A stable error on one specific case usually points to a gap in the examples: check whether that case is covered and whether adding a labelled one changes the answer.

The typical traps

• Asking for “the correct format” or fixing the output with post-processing: it hides the problem.
• Examples that are all easy, or all from the same category.
• Examples without reasoning, which the model copies literally without understanding the boundary.
• Measuring on the same cases that are in the examples.

On the exam

If the scenario says the instructions are detailed but the output is inconsistent, the answer involves examples. Then pick the option that best targets the failure: one example per input pattern for format, ambiguous cases with their why for judgement, examples of different structures for an extraction that leaves fields empty.