claudio.cool
CCAR-PLessons

CCAR-P · Developer Productivity & Operational Enablement

AI-assisted development workflows and incidents, plan before implementing, reproducible CI and rate limits

⏱ 3 min readDraft

Three scenes from the same week. A developer asked Claude Code to migrate forty files to a new payments client in a single request and got inconsistent changes that nobody verified. The automated pull request review in CI gives different results depending on the runner. And after launching a feature that makes many calls in parallel, the app gets 429 errors, retries them instantly and the incident gets worse.

Large changes: explore, plan, implement, verify

A giant request in a single step leaves Claude guessing at the scope with no way to know whether it finished well. The recommended workflow separates the stages:

  1. Explore in plan mode: Claude reads the code and answers questions without modifying anything.
  2. Plan: Claude proposes which files change and in what order; a person reviews and corrects the plan before any code is touched.
  3. Implement in steps, checking against the plan.
  4. Verify with something that passes or fails: the test suite, the build, a linter. It’s the difference between a session you have to watch and one that corrects itself.

Subagents help investigate without filling the main context, and it’s worth clearing the context between unrelated tasks. If the change is small and fits in one sentence, the plan is unnecessary. For migrations of hundreds of files, you can fan out with a script that calls Claude per file, trying two or three first.

Reproducible CI

In CI, Claude Code runs in non-interactive mode with claude -p. The problem in the scenario is that, by default, that run loads whatever it finds on the machine: hooks, plugins, MCP servers, memory and the runner user’s CLAUDE.md. Each differently configured runner gives a different result.

Bare mode, with --bare, starts without reading any of that. Whatever the job needs is passed explicitly:

  • --allowedTools to pre-approve only the tools the job needs;
  • --settings and --mcp-config for configuration and servers;
  • --append-system-prompt for the review instructions;
  • --output-format json so the pipeline gets a result it can process.

That way the job is the same on any runner. Skipping all permissions so nothing stalls, or configuring each machine’s ~/.claude by hand, doesn’t solve reproducibility.

API incidents

Signal What it means What to do
429 with retry-after a rate limit was exceeded wait as long as the header says, exponential backoff with jitter, cap concurrency
429 without retry-after, from the spend cap the monthly spend limit was reached retrying won’t help: the limit has to be raised
529 the API is overloaded retry with backoff
500 internal error backoff; if it persists, support with the request ID
400 the request is wrong fix it; retrying changes nothing

Immediate retries multiply traffic exactly when there’s too much of it. Jitter keeps every client from retrying at the same instant. The official SDKs already retry with backoff and honor retry-after. To prevent it: ramp traffic up gradually and use caching, because on most models tokens read from the cache don’t count toward the input limit.

The usual traps

  • A longer prompt with all forty files and every rule.
  • Asking Claude to review its own diff at the end instead of running the tests.
  • Starting an interactive session from the CI script.
  • Raising immediate retries to twenty.
  • Moving to another model “because it has a different limit” or treating the 429 as a provider outage.

On the exam

If a large change came out inconsistent, look for exploring and planning with review, implementing in steps and verifying with tests. If CI gives different results per machine, look for claude -p with --bare and everything passed through flags. If there are 429s and immediate retries, look for exponential backoff with jitter, retry-after and a concurrency cap.

Quick question

A practice exam question on this domain. It counts towards your progress.

A media company added an automated pull-request review step to its CI pipeline: a job runs claude -p with the diff piped in and posts the result as a comment. The job runs on a pool of self-hosted runners that developers also use for other tasks. Reviews are inconsistent: on some runners the job picks up a hook from a developer's home directory that rewrites prompts, on others it connects to an MCP server declared in the repository, and one run stalled waiting for a permission nobody could grant.

The team wants every run to behave the same way, with only the tools the review needs, and output that the pipeline can parse reliably.

How should the team configure the CI job?

Practise this domain in a practice exam →