Retries, idempotency and bounded concurrency; refactor and review in small steps

A mobile app sends an essay to be graded. The backend calls Claude, saves the feedback and deducts a credit, but it takes about twenty seconds. At fifteen, the client assumes it failed and resends the request: the student gets two sets of feedback and pays two credits. None of this is a model problem. It’s everyday software engineering, and the exam asks about it with Claude scenarios.

REST, JSON and what can be repeated

In a REST API, the method tells you what repeating a request does. A GET that reads can be repeated without care. A POST that creates something, charges or sends an email can’t: two identical POSTs are two effects.

When the client may resend a POST (on timeout, on reconnect, because a user taps twice), you have to make it idempotent:

1. The client generates a unique idempotency key per operation and sends it with every attempt.
2. The server stores the key along with the operation’s state.
3. If the same key arrives again, it returns the stored result (or “in progress”), without calling Claude or charging again.

With JSON, the usual applies: validate what comes in and what goes out against a schema, and don’t trust a field to exist because “it was always there”.

Asynchrony and concurrency with a limit

Analyzing three hundred reviews one by one takes minutes. Launching them all at once with an unbounded gather ends in a wave of 429s. The answer is in between:

• Asynchronous programming with the SDK’s async client, so you don’t block while waiting for each response.
• Bounded concurrency: a limit on in-flight requests, for example with a semaphore of ten, tuned to your rate limits.
• Backoff with jitter for the occasional 429 that still shows up (the SDK’s retries already do this; don’t add another loop on top).

And if nobody is waiting for the work, the option is the Message Batches API, covered in the next lesson.

Version control and code review

Code generated by AI follows the same path as any other code:

• work on its own branch, never a direct push to a branch that deploys automatically;
• a pull request with tests running in continuous integration (CI);
• a human code review before it reaches main.

The fact that an assistant wrote the change doesn’t make it more or less trustworthy: what makes it trustworthy is tests and review.

Large refactorings, in small steps

A refactoring changes structure without changing behavior. The risk shows up when a sixty-file PR mixes renames with a hidden logic change, such as different rounding. To keep it reviewable:

1. Before touching anything, tests that pin down current behavior.
2. First, the mechanical changes (rename, move, extract) in small PRs that don’t alter results.
3. Then, each behavior change separately, with its new test and a description that explains it.

Claude Code helps a lot with the mechanical step; splitting into PRs is what lets the change that matters be seen.

The typical traps

• Retrying a POST with side effects without an idempotency key.
• Parallelizing without a limit, or fixing 429s with more retries instead of less concurrency.
• Adding your own retry loop on top of the SDK’s.
• Pushing generated code directly to a branch that deploys automatically.
• A giant PR that mixes refactoring with logic changes.

On the exam

If there are duplicated effects from resends, look for the server-side idempotency key. If it’s slow in series and fails in parallel, look for the async client with a concurrency limit. If the problem reached production through unreviewed code, look for branch, pull request, CI and human review. And if a behavior change hid inside a refactoring, look for the option that separates the mechanical part from what changes results.