Connected Patterns: Memory That Prevents Drift Without Becoming Bloat
“Memory is not a dump. Memory is a set of decisions about what must not be lost.”
Agent builders talk about memory as if it were a magical feature: add memory, get better agents.
Featured Console DealCompact 1440p Gaming ConsoleXbox Series S 512GB SSD All-Digital Gaming Console + 1 Wireless Controller, White
Xbox Series S 512GB SSD All-Digital Gaming Console + 1 Wireless Controller, White
An easy console pick for digital-first players who want a compact system with quick loading and smooth performance.
- 512GB custom NVMe SSD
- Up to 1440p gaming
- Up to 120 FPS support
- Includes Xbox Wireless Controller
- VRR and low-latency gaming features
Why it stands out
- Compact footprint
- Fast SSD loading
- Easy console recommendation for smaller setups
Things to know
- Digital-only
- Storage can fill quickly
In practice, memory is where many agent systems break.
If memory is too thin, the agent forgets constraints and repeats work.
If memory is too thick, the agent carries stale assumptions, bloats context, and becomes slow and confused.
The goal is not maximum memory. The goal is correct memory.
Correct memory stores what must remain true across time, and recomputes what can be derived safely.
Why Memory Is Harder Than It Looks
Humans hold memory with judgment. We remember what matters and forget what does not.
Agents need that judgment encoded.
Without explicit policies, memory becomes:
- A transcript stuffed into context until the model loses the thread.
- A summary that overwrites nuance and introduces errors.
- A database of “facts” that are no longer true.
- A grab bag of notes that the agent cannot prioritize.
Memory needs structure, not volume.
The Three Layers of Agent Memory
Most reliable systems use three distinct layers.
- Working context: the short window the agent uses to think right now.
- Durable state: the structured snapshot that persists across steps and restarts.
- External knowledge: systems the agent queries when needed, such as search or databases.
When you separate these layers, you can keep each one healthy.
Working context stays small and relevant.
Durable state stays structured and checkable.
External knowledge stays authoritative and refreshable.
What Belongs in Durable State
Durable state should store only what the agent must not forget.
Examples:
- The target statement in one sentence.
- The acceptance criteria that define done.
- Constraints such as budgets, safety boundaries, and required approvals.
- Decisions already made, including why they were made.
- The current plan and what has been completed.
- Open questions that require human input.
This is not “everything the agent saw.” It is the spine of the run.
What Should Usually Be Recomputed
Many items feel important, but become dangerous when stored long-term.
Common candidates for recomputation:
- Facts that change frequently.
- Summaries of external pages that may update.
- Derived conclusions that depend on tools, versions, or environment state.
- Rankings, counts, and metrics that can drift.
If the information is cheap to retrieve and likely to change, storing it in long-term memory invites staleness.
The Memory Decision Table
A simple way to decide is to ask two questions:
- Will this still be true tomorrow?
- Is it expensive to compute again?
| Information type | Store in durable state | Recompute when needed | Why |
|---|---|---|---|
| Target and acceptance criteria | Yes | No | If lost, the run drifts |
| Safety boundaries and approvals | Yes | No | Protects against unsafe actions |
| Tool outputs with timestamps | Sometimes | Often | Store only if needed, include “as of” time |
| External facts without dates | No | Yes | Too easy to become stale |
| Decisions and rationale | Yes | No | Prevents contradiction and rework |
| Intermediate drafts | Sometimes | Sometimes | Store if needed for audit, otherwise regenerate |
| Metrics and counts | No | Yes | Refresh from authoritative sources |
This table is not perfect, but it prevents the most common mistakes.
Compaction: Turning Progress Into a Stable Snapshot
Long runs require compaction. The agent cannot carry everything forward.
Compaction works when it preserves:
- The target and constraints.
- The decisions that shape future steps.
- The evidence that supports key claims.
- The remaining plan and open questions.
Compaction fails when it produces a vague summary that sounds coherent but loses the sharp edges. Sharp edges are the constraints that keep the run on track.
A good compaction snapshot reads like a run sheet, not like a story.
Freshness as a First-Class Field
If you store any fact that can change, attach freshness metadata:
- as_of timestamp
- source identifier
- retrieval method
- confidence or verification status
Then add a policy:
- If as_of is older than the task’s tolerance, refresh.
- If the source is missing, downgrade trust and re-verify.
- If a later tool output contradicts stored memory, surface the conflict.
This turns stale memory from a silent bug into a visible event.
Preventing Memory From Becoming a Second Brain of Noise
Memory systems often collapse because they accept everything.
A better approach is memory admission control:
- Only store items that match a memory schema.
- Require a reason for storage: which future step needs it.
- Cap memory size and evict items that are redundant or unverified.
- Prefer structured fields over prose notes.
Admission control is how you keep memory from turning into a pile of debris.
Memory and Trust
The agent should trust different memory fields differently.
- Trust constraints and approvals highly, because they are system inputs.
- Trust decisions moderately, because they may require review.
- Trust external facts only if they are dated and sourced.
- Trust summaries least, because they can be wrong in subtle ways.
If you encode these trust tiers, routing and verification become much easier.
A Minimal Durable State Schema
A durable state snapshot should be predictable enough that tools and validators can read it.
A minimal schema can include:
- target: one sentence
- done_criteria: a short list of checkable criteria
- constraints: budgets, permissions, dependencies
- decisions: decision log with timestamps and rationale
- plan: current step list with status and evidence pointers
- open_questions: items that require human input
- risks: known risks and mitigations
- artifacts: links or IDs for drafts, patches, or reports
If you keep these fields stable, compaction becomes easy because you are updating a form rather than rewriting prose.
The Difference Between Memory and Cache
Some teams store tool outputs in memory to avoid rework. That is useful, but it is not always memory.
Cache is for speed. Memory is for correctness.
If a value is cached, it should include a freshness policy and an invalidation rule. Otherwise, the agent will treat yesterday’s output as today’s truth.
A simple invalidation rule is enough:
- Invalidate cached tool outputs when the environment changes.
- Invalidate when the run crosses a time threshold.
- Invalidate when a later tool output contradicts the cached value.
Cache becomes safe when it is explicit and revocable.
A Memory Failure Story That Shows Why Policies Matter
Consider an agent that helps manage a long-running migration. Early in the run, a human decides that a certain subsystem is out of scope. The agent stores that decision in a free-form summary.
Two days later, the agent compacts context again and rewrites the summary. The “out of scope” sentence is dropped because it seemed less important than other notes. The agent begins proposing changes to the subsystem, and the team wastes hours correcting it.
This is not a “smartness” failure. It is a memory policy failure.
The fix is durable state discipline:
- Store scope constraints in a dedicated field.
- Treat scope constraints as high-trust inputs.
- Require compaction to preserve constraint fields verbatim.
- Add a guardrail that blocks actions that violate scope constraints.
Once you do this, the same failure stops happening.
Recompute Patterns That Keep Memory Lean
Recomputing does not have to be expensive. It can be a structured move:
- Refresh: re-run the same tool call with the same inputs, then compare outputs.
- Diff: compare current output to cached output and store only the difference.
- Verify: cross-check a stored fact against an authoritative source.
- Summarize: compress a large artifact into a stable state field, but retain the artifact ID for audit.
These patterns keep the agent moving without turning memory into a landfill.
The Payoff: Less Drift, Faster Runs, Better Debugging
When memory is structured, agents stop repeating the same work. They stop contradicting earlier decisions. They stop bloating context until they lose coherence.
You also gain something most teams do not expect: debuggability.
A durable state snapshot lets you replay a run, understand why decisions were made, and fix system-level problems instead of chasing “model moods.”
Memory choices determine the personality of an agent system. Store the invariant spine, recompute the moving surface, and your agents become steadier with time instead of more confused. When memory is disciplined, reliability stops being a hope and becomes an engineered outcome.
Keep Exploring Reliable Agent Workflows
• Context Compaction for Long-Running Agents
https://ai-rng.com/context-compaction-for-long-running-agents/
• Agent Checkpoints and Resumability
https://ai-rng.com/agent-checkpoints-and-resumability/
• Preventing Task Drift in Agents
https://ai-rng.com/preventing-task-drift-in-agents/
• Latency and Cost Budgets for Agent Pipelines
https://ai-rng.com/latency-and-cost-budgets-for-agent-pipelines/
• Monitoring Agents: Quality, Safety, Cost, Drift
https://ai-rng.com/monitoring-agents-quality-safety-cost-drift/
