Skip to content
NTPU AI Learn

AI 302 · Module 6

Context Compaction & Long-Running Agents

The longer an agent works, the more context piles up. How do we avoid bloat and accumulating errors?

Start Section 1
  • Duration50 min
  • LevelArchitect
  • PrerequisiteAI 301 recommended, plus Modules 1 to 5 of this course
0 / 5 completed
Sync my progressJoin a session to keep progress across devices and share it with your instructor

Info

After you join, your completed lesson numbers are sent to your instructor together with your learner code, so they can see your progress. Use the codes your instructor gave you, and do not enter your name, company secrets or other personal data. Progress is kept with the rest of that session's data.

Learning objectives

  • Explain context growth and context budget in a long-running agent
  • Describe context compaction, summarization and building a fresh working context, and say what compaction is not
  • Separate persistent state from ephemeral state, and list what must never be dropped
  • Recognize stale context and context drift, and explain context refresh and memory handoff
  • Complete Activity 6 and produce a Context Compaction Policy

Context growth and compaction

10 min

Why long-running agents run into context limits

A short task fits easily in an agent context. A long-running agent keeps adding to it: instructions, tool results, drafts, corrections, human replies. Over time the context grows until it is expensive, slow, noisy and can no longer fit at all.

Core question

The longer an agent works, the more context piles up. How do we avoid information bloat and accumulating errors?

Key terms
TermPlain meaning
Long-running agentAn agent that works across many steps, sessions or hours instead of one short exchange
Context growthThe steady increase of information the agent has to carry
Context budgetHow much context the agent can usefully hold, given size limits, cost and attention
Working contextWhat the agent actually sees when it takes its next step
Historical contextThe full record of what already happened, which is larger than the working context
Simulated example

A simulated research agent has run for a long session. Its context now holds 40 tool results, three abandoned drafts, and one important instruction the human gave at the start. The instruction is buried, and the agent begins to ignore it.

Key takeaway: Context grows with every step. A long-running agent needs a deliberate way to keep its working context small and relevant.

10 min

Context compaction: what it is and what it is not

Compaction reduces execution history while keeping what the next steps depend on. It is a selective process, and it has a clear lifecycle.

Context lifecycle
  1. Execution History: everything that has happened so far
  2. Select Important State: decide what the next steps depend on
  3. Summarize / Compact: shorten it into a state summary
  4. Preserve Evidence: keep source references for what was summarized
  5. Build Fresh Working Context: assemble a clean context to continue

Info

Text version of the diagram: five steps in a line. Execution History goes to Select Important State, then to Summarize / Compact, then to Preserve Evidence, and finally to Build Fresh Working Context, which is what the agent sees next.
Compaction, compared
Compaction is notCompaction is
Deleting old information at randomPreserving decision-relevant state while removing unnecessary execution history
Shortening everything equallyChoosing carefully what to keep, summarize, archive or discard
Losing the trailKeeping references so details can be retrieved later
Check yourself

Which statement best describes context compaction?

Which statement best describes context compaction?

Key takeaway: Compaction is selective: keep what decisions depend on, shorten the rest, and keep evidence references.

What to keep and what to refresh

10 min

What must survive compaction: persistent and ephemeral state

Before compacting, sort the state. Persistent state must remain true across the whole run and survive compaction. Ephemeral state is only useful for a moment and can be dropped once its result is captured.

Persistent versus ephemeral state
KindMeaningSimulated example
PersistentNeeded for the run to stay correct and accountableThe goal, an active constraint, a human decision
EphemeralUseful only for one step, safe to drop after its result is recordedA raw tool response that has been summarized, a discarded draft
What should survive compaction
ItemWhy it must not be dropped
GoalWithout it the agent drifts
Current stateThe next step depends on where the run stands
Active constraintsDropping a limit lets the agent break it
Key decisionsForgetting them causes repeated or reversed choices
Unresolved issuesOpen problems must not silently vanish
Evidence referencesClaims must stay traceable to their sources
Pending actionsA forgotten pending action is never done, or is done twice
Important human instructionsThe person stays in charge; their instruction must not be lost

Warning

Never drop a human instruction, an active constraint or a safety rule during compaction. Losing them can make an agent act outside what a person approved.
Check yourself

During compaction, which item is the most dangerous to lose?

During compaction, which item is the most dangerous to lose?

Key takeaway: Keep goal, state, constraints, decisions, open issues, evidence, pending actions and human instructions. Drop only what is ephemeral.

10 min

Stale context, context drift, refresh and memory handoff

Old context does not stay true. Stale context is information that used to be correct and no longer is. Context drift is the slow shift of an agent away from its original goal as summaries of summaries pile up. Both make a confident agent wrong.

Keeping a long run healthy
IdeaPlain meaning
Context refreshRebuilding the working context from the goal, state and sources, instead of trusting an ever-longer history
Memory handoffPassing the essential state to a new session or agent so work can continue without the whole history
Evidence retentionKeeping the evidence behind a claim, so it can be checked again
Source referencesPointers to where information came from, so details can be retrieved later instead of carried along
Simulated example

After a compaction, a simulated agent summary says "the vendor list is approved". A human later removed one vendor. Because the summary kept a source reference and a timestamp, a refresh before the final step catches that the list changed.

Info

Text version of the idea: at planned points, do not just add to the old context. Rebuild it from the goal, the current state and the sources, and check that key facts still hold.

Info

Example only, not required: a stateful framework such as LangGraph keeps run state separate from the messages the model sees, which makes a refresh easier to build. The concepts apply to any framework.
Check yourself

Why keep source references when a long history is compacted?

Why keep source references when a long history is compacted?

Key takeaway: Refresh working context from goal, state and sources. Keep evidence references so summaries can be checked.

Practice and deliverable

10 min

Activity 6: Compact the Context and the Context Compaction Policy

Take a simulated long execution history, or one from your own workflow design. Sort each item by what should happen to it so the agent can keep working from a clean working context.

  1. List the history
    Write 10 to 15 items from the execution history: instructions, tool results, drafts, decisions, open issues.
  2. Keep
    Mark items that must stay in the working context, such as the goal, active constraints and human instructions.
  3. Summarize
    Mark items to shorten into a state summary, and write the summary.
  4. Archive and retrieve later
    Mark items to store outside the working context, with a source reference so they can be retrieved when needed.
  5. Discard
    Mark ephemeral items that can be dropped, and say why it is safe.

Warning

Privacy reminder: use simulated or already-public information. Do not enter company secrets, client personal data or unauthorized sensitive information.

Info

Deliverable: a Context Compaction Policy. It states when to compact, what must be kept, summarized, archived and discarded, how evidence is retained, and when the working context is refreshed. Keep it for your Agent Harness & Reliability Canvas.
Check yourself

In your history, a human wrote "only use approved sources". Where does it go?

In your history, a human wrote "only use approved sources". Where does it go?

Key takeaway: Sort history into keep, summarize, archive and discard, protect human instructions and constraints, and record it in your Context Compaction Policy.

Save as PDF

What next?