Fight context rot with garbage collection

Context management with agents is similar to memory management with traditional computer programs. You only have so much of it until the process crashes or stops working. Most modern programming languages like Python and Java use precise garbage collection to manage memory effectively. AI agents do not have an equivalent mechanism for context.

Background agents of the future

I believe the agents of the future will be different from the chatbot and human-in-the-loop agents common today. You'll give an agent a task, some general pointers, an array of tools, and it will just go figure it out. Real, meaningful work in the background, autonomously, over longer time horizons.

We do not see agents like this today because when people try to build them, they do not work very well. And they do not work very well because of the context rot problem.

Context rot

The context rot problem is where agents lose effectiveness as the reasoning turns increase. This happens because the context window fills up with old tool call results, stale reasoning, dead-end explorations, etc. This context that was useful three steps ago is now just noise. The agent's reasoning quality degrades, it starts hallucinating, it ignores user instructions, it repeats work it already did. And the whole time, you're paying for every token on every conversation turn (even if the inference provider caches input tokens).

The most popular way to address the context rot problem is the fill-and-compress approach. Fill the window and then summarize and compress the context to free up space. This is essentially a primitive form of garbage collection, but it does not work very well. Summarization is lossy. The model has to decide what to keep and what to drop, and it can easily discard details that turn out to be critical three steps later. The compression pass itself is also slow and expensive. It's an inference call to save on future inference calls.

More from us on context rot

A quick primer on garbage collection

In languages like Python or Java, when a function finishes executing, the local variables it created go out of scope. The runtime's garbage collector reclaims that memory so the program can reuse it. The caller gets back the return value and nothing else. All the internal state that the function needed to do its work is gone.

Better garbage collection

Let's use an example to see what better garbage collection might look like for an agent.

Imagine a patent infringement analysis agent. The agent will generally need to accomplish the following steps:

• Read and classify a patent
• Fan out a series of search calls to find prior art
• Prune search results by relevance
• Conduct deep comparisons against each remaining candidate
• Perform a secondary deep dive on the closest matches to highlight key differentiators
• Audit all external references for accuracy
• Compile a final pass/fail report

Each of those major steps might internally require dozens of individual tool calls and reasoning turns. But from the workflow's perspective, they should be thought of like function calls. They take an input, do a bunch of work, and produce a result. After the prior art search step finishes, the agent doesn't need the raw content of every search result sitting in its context. It needs the pruned shortlist. That's the return value. After the deep comparison step, it doesn't need every side-by-side analysis. It needs the summary of which patents are closest and why.

For the same patent analysis workflow, here is what the context window looks like for the fill-and-compress approach (today's agents) vs. the prune-as-you-go approach (agents with better garbage collection):

These context window profiles are very different. For prune-as-you-go, the overall footprint grows very gradually over the life of the task and effectively extends the context window when compared to the fill-and-compress approach. Pruning context at logical boundaries keeps the agent focused and minimizes the risk of over-compression that you experience with the fill-and-compress approach.

What about subagents?

When agents start choking on context, the popular pattern is to delegate work to subagents. This can work, but it comes with a lot of baggage. Now you're managing agent-to-agent communication, error handling, context passing, etc. In most situations where people turn to subagents, I think they do so because the parent agent's context can't survive the bloat. Subagents are a workaround for an ineffective garbage collector.

What about skills?

Skills are pre-written instructions, templates, and functions that guide an agent through specific subtasks they may run into. I think of them as tools on steroids. We are already seeing how skills make agents faster, more efficient, more reliable, and more deterministic by encoding best practices up front. They are genuinely powerful and I'm really excited to see what skills unlock for the future of agents.

At the same time, skills require pre-cognition of specific hurdles the agent will face so they can be encoded up front. For novel, complex, long-horizon tasks - where there is likely to be uncertainty along the way - it may not be possible to cover all your bases with skills. Conversely, skills can also compound the context rot problem. The inputs and outputs of skills can be lengthy, and when the agent chains multiple skills together in a multi-hop workflow, the context bloats fast. Skills are an excellent tool to fight the context rot problem, but they are not a silver bullet.

Looking forward

I do not know the perfect technical solution to the context rot problem. I think it will be a combination of a lot of things. But what I do feel confident in is that the most productive way to frame this problem is through the lens of classical garbage collection. The agent systems that figure out how to roll up context into clean inputs and outputs naturally, the way a Python script handles function calls and memory, will be the simplest to build, the cheapest to run, and the most reliable.