Spotting bias in responses

A recruiting team receives thirty applications and asks Claude for a five-line summary of each candidate, with “strengths and concerns.” A second recruiter reads the summaries next to the CVs and notices something: several women are described as “collaborative” and “supportive,” and men with equivalent track records as “driven” and “strategic.” A two-year employment gap shows up as a concern for some candidates and, for others with the same gap, isn’t mentioned at all.

Each summary on its own looks reasonable. The problem appears when you compare them.

What bias in an output is

Bias is a systematic difference in how similar cases are treated, one that the facts of the case don’t explain. Models learn from texts written by people, and can reproduce associations from those texts: which adjectives go with whom, what counts as a risk and for whom.

A hallucination is detected by comparing the output with the source. Bias, by contrast, is detected by comparing outputs with each other.

Signals to look for

• Inconsistent treatment: the same fact (an employment gap, a change of industry, an age) is flagged as a problem in some cases and ignored in others.
• Stereotyped language: different adjectives for equivalent profiles, following gender, age, origin or any other trait unrelated to the task.
• Irrelevant mentions: the summary comments on something personal that has nothing to do with the role or the decision.
• A single perspective: in an analysis or a recommendation, only one side’s arguments appear, or a group of customers is always described in the same negative tone.

A practical check: take two cases with equivalent facts and read their outputs side by side. If the description changes more than the facts do, there is something to review.

How to fix it

The open request (“strengths and concerns”) leaves it to the model to decide what to look at in each case. The fix is to remove that leeway:

1. Define consistent criteria before processing the cases. In hiring, job-related criteria: years in complex sales, managing large accounts, required languages.
2. Apply them equally to everyone. The same format and the same questions for every candidate: “for each criterion, what evidence is in the CV; if there is none, write ‘no evidence’.”
3. Regenerate all the outputs, not just the ones that caught your eye.
4. Check against the source: compare a sample of summaries with the CVs.
5. Decisions about people are made by a person. The summary helps you read faster; it doesn’t build the shortlist (more on this in the governance domain).

This logic applies beyond hiring: performance reviews, complaint analysis by region, customer segmentation. Whenever many similar cases are described, use consistent criteria and compare across cases.

Common traps

Option that shows up	Why it’s wrong
Adding “be fair and unbiased” and regenerating	It’s an intention, not a criterion; it doesn’t say what to look at
Asking Claude which candidates it described with bias	The same model is not an independent check
Leaving it because “only the words change”	The words influence who moves to the next stage
Fixing by hand the summaries you noticed	The rest are still made with the same open request
Letting the summary decide the shortlist	Takes the person out of a decision about people

On the exam

The typical clue is a comparison: similar profiles with different descriptions, or the same fact treated differently depending on the case. The scenario almost always makes clear that the prompt was open-ended. Look for the option that replaces the open request with consistent, task-related criteria applied to everyone, and checks against the source. Rule out the ones that rely on a well-meaning sentence, on the model’s self-assessment or on things “looking fine overall.”