CCAR-F · Context Management
The context window and how to manage it
The context window is everything the model can read in a single call: system instructions, conversation history, tool results and documents. If something is not in the window, the model does not know it. That is why every API request has to carry the full conversation history.
Why it fills up
In an agent, every step adds text, and tool results are usually the largest part. An order lookup can return 40 fields when only 5 matter for a return. The fix is to trim the result to the relevant fields before it enters the history, not once it has already piled up.
What gets lost when summarizing
Progressive summarization compresses the history again and again. The risk is that concrete data turns vague: “$1,250 on March 3” becomes “a recent refund”, and “they promised me an answer by Friday” disappears.
What you do instead is pull the transactional data (amounts, dates, order numbers, statuses, what the customer expects) into a persistent case facts block that goes in every prompt, outside the summarized history. If the session has several issues at once, each one keeps its structured data in that separate layer.
Position matters
With long inputs you get the lost in the middle effect: the model handles the beginning and the end well, but may omit what sits in the middle. To mitigate it:
- put a summary of the key findings at the beginning;
- organize the detail into sections with explicit headers.
Long explorations
In a long session over a large codebase, the model starts answering with “typical patterns” instead of the specific classes it found earlier. It helps to have the agent write key findings to a scratchpad file and consult it later, and to use /compact when the context fills up with exploration output.
Prompt caching cuts cost and latency, but it does not free up room in the window.
On the exam
Look at which piece of data was lost and where it was. If it is figures or dates that got diluted by summarizing, the answer is a case facts block; if the finding was in the middle of a long input, reorder it; if the context fills up with fields nobody uses, trim the tool results.
An agent has been running for hours and the conversation is approaching the context window limit. What is the best way to keep it working without losing what matters?
Quick question
A practice exam question on this domain. It counts towards your progress.
A fintech company is building an internal Claude agent that helps analysts investigate unusual account activity. The agent has several tools that query external databases: transaction history, account events and an audit log service. Each tool simply returns whatever the underlying API sends back.
In practice, the audit log tool often returns raw JSON logs of more than 30,000 tokens, with nested request headers, internal IDs and timestamps for every event. A typical investigation calls three or four tools in a row, so the context window fills up quickly.
The traces show a clear pattern: the first steps of a multi-step task go well, but later steps start to fail, with the agent losing earlier instructions or repeating calls it has already made. For most tasks the agent only needs a handful of fields from each log entry, such as the event type, amount, account and time.
The team has eight engineers and a compliance review deadline at the end of the month. One developer already tried raising the request timeout, which made no difference.
What should you do to stop tool results from overwhelming the context window?