claudio.cool
CCDV-PLessons

CCDV-P · Advanced Agent Architecture and Orchestration

Context and memory policies for long-running agents

⏱ 3 min readDraft

A technical-support agent is two hundred turns into a session. The window is at 90%, each turn costs more than the last and the agent starts forgetting the policy it was given at the start. On top of that, the team wired in forty tools and there are already 20,000 tokens of definitions before the customer says a word.

Policy options, and when each fits

Policy What it does When Risk
Context compaction summarizes old history and continues from the summary long conversations with a continuous thread the summary drops exact data (amounts, dates, commitments) unless told to keep them
Context editing removes content no longer needed, such as old tool results agents with many bulky calls deleting something that’s needed again; it must be re-fetchable
Persistent memory stores facts outside the window and looks them up when needed information that must outlive the session or the trim memory that fills with junk or goes stale
Subagents take a stretch of work in their own window and return a summary noisy exploration badly defined handoff contract

Claude Code compacts automatically when nearing the limit and offers /compact to do it manually; the API and Agent SDK have equivalent mechanisms you should check in the current documentation. What matters for the decision isn’t the mechanism’s name but what gets lost. Data that must not be lost (constraints, commitments, identifiers) goes in a pinned block or in memory, not in the free-form summary.

Prompt caching doesn’t free space: it lowers cost and latency of the repeated prefix, but that prefix still occupies the window.

Changing instructions without breaking the cache

Prompt caching works on an exact prefix: tools, then system prompt, then messages. If mid-conversation you edit the system prompt or change the tool list, you invalidate the cache for everything after it and pay for the prefix again. When an instruction changes during the session (a new mode, a restriction), it’s cheaper to append it as a new message at the end than to rewrite the beginning. Fixed, long content goes on top; what changes goes at the bottom.

Large tool catalogs out of the window

Forty tool definitions loaded every time cost tokens on every turn and muddle tool choice. The answer is tool search with deferred loading: the model gets a search capability and pulls in only the definitions it needs at that moment. The API has a tool-search tool and a marker to defer loading; check the current names in the documentation before implementing. With MCP, same idea: don’t expose every server to every agent.

The usual traps

  • Summarizing without saying what to preserve verbatim.
  • Trusting the cache to “fix” a full window.
  • Changing the system prompt every turn (and losing the cache every turn).
  • Loading every tool “just in case”.
  • Persistent memory with no rule for what enters, what gets updated and what gets deleted.

On the exam

Look at which limit is about to break: the window (compact or edit, with critical data protected), cost (a well-ordered cache and less context), or the tool catalog (deferred loading). Discard the option that swaps in a “bigger” model without addressing context design.

Quick question

A practice exam question on this domain. It counts towards your progress.

A finance team runs an invoice-triage agent that decides, for each incoming invoice, which tools to call: vendor lookup, duplicate check, purchase-order match, and approval routing. After four months the team analyzes 50,000 runs. In 94% of them the agent follows exactly the same four steps in the same order. The remaining 6% involve odd invoices where the agent explores, asks for clarification, or consults several systems.

Average cost per run and p95 latency are well above the targets, and the 94% group accounts for most of the spend.

What should the team do?

Practise this domain in a practice exam →