Context, State and Memory

These three words sound similar, but they answer three different questions:

• Context: What information can the model see during this step?
• State: What is currently true about this task?
• Memory: What information was saved so it can be used later?

Saved memory ─┐
Current state ├→ context for this model call → decision → updated state
New message  ─┘                                      ↓
                                             save useful memory?

What You Will Learn

• How context, state, and memory differ even when they contain some of the same facts.
• Where each one is stored and when it enters the agent loop.
• Why a model’s context window is not the same thing as permanent memory.
• How short-term, long-term, semantic, and episodic memory can be implemented.
• What to save, what to retrieve, what to forget, and how to protect private data.

How This Appears in a Current Agent

Anthropic describes context engineering as deciding which instructions, tools, retrieved data, message history, and current state should occupy the model’s limited context on each step. The agent may have far more stored information than can—or should—be shown to the model at once. (Anthropic, Effective Context Engineering for AI Agents)

So memory is not useful merely because it was saved. It becomes useful when the system retrieves the right memory and places it into the right model call.

Module 3 introduced state and memory as separate boxes in the agent architecture and asked you to hold them apart without fully explaining why the distinction mattered. It’s time to give that distinction the real attention it deserves, because, confusing these three concepts — context, state, and memory — is one of the most common sources of confusing agent behavior once you start building for real. “Why did it forget that?” and “why is it including information it doesn’t need?” are almost always, underneath the surface, questions about which of these three things happened — or didn’t.

Three different things, made concrete

Let’s ground this in something you can hold a precise mental picture of: an agent whose job is to book a flight for a customer.

As the agent works through the task — checking available routes, confirming dates, finding a flight that matches — it builds up a , evolving understanding of where things currently stand:

destination
travel dates
passenger
available flights
selected flight
payment status
booking status

This is state — the agent’s accumulated, in-progress record of what it has established so far, specific to this one booking task. It starts empty. It fills in as the loop proceeds — destination gets set once the customer specifies where they’re going, available_flights gets populated once a search tool runs, selected_flight gets set once a choice is made. Once the booking is complete (or abandoned), this entire structure is discarded. It has no life beyond this one task.

Context is a different thing: it’s everything handed to the model for one specific reasoning step. Say the agent is at the point of deciding which flight to recommend — the context for that one decision might include the current state (destination, dates, passenger preferences), the list of available flights just retrieved, and the most recent thing the customer said. Context isn’t stored anywhere as its own persistent object.

It’s assembled fresh, each time, from whatever’s currently relevant — typically some combination of state, retrieved information, and the latest observation. The next reasoning step, a moment later, gets a new context — not necessarily identical to the last one, even though it’s drawing from much of the same underlying state.

Memory is the third piece, and it’s about something that survives beyond this one task entirely. If this same customer books another flight next month, and the agent recalls “this customer always prefers aisle seats and has flagged a peanut allergy on file,” that’s memory — deliberately retained, persisting across sessions in a way state simply doesn’t.

Why the distinction is worth this much attention

Here’s a way to hold all three at once that might make it click: think of state as a whiteboard you’re actively filling in for the specific job in front of you right now — erased and reset the moment that job ends. Think of context as whatever’s in your field of view at any single instant while you work — some of it from the whiteboard, some of it from what someone just told you, assembled fresh each time you glance up.

And think of memory as a filing cabinet you deliberately put things into when you decide they’re worth keeping around for next time — separate from the whiteboard, and still there long after this particular whiteboard has been wiped clean.

The reason this distinction matters practically, not just academically: each of these three has a different lifetime, and building an agent means making real, deliberate engineering decisions about each one separately. Getting state wrong means an agent loses track of what it already established mid-task.

Getting context assembly wrong means a model either doesn’t see something it needs, or gets buried under things it doesn’t — a direct connection to your RAG course’s lesson that more retrieved content isn’t automatically better content; the same discipline applies to assembling context from state and memory, not just from retrieval. And getting memory wrong means an agent either forgets things it should have remembered, or persists things that should have stayed scoped to one task and never survived past it.

Why state is necessary — go back to Module 4

You’ve already watched state do real work, back in Module 4’s full loop walkthrough, even though we didn’t name it explicitly at the time. By iteration 3 of that support-agent run, the agent needed to know what iteration 2 had discovered — the payment-history mismatch — in order to reason correctly about what iteration 3 should even check. Without state persisting that discovery between iterations, each step of the loop would effectively be starting from nothing, re-deriving (or worse, simply not knowing) what earlier steps had already established.

This is precisely why state has to exist as a real, distinct mechanism, not something left implicit. A multi-step task, by definition, depends on later decisions being informed by earlier discoveries — and state is specifically the thing that carries those discoveries forward.

Why memory becomes necessary once tasks stop being isolated

Now imagine that same customer from our support agent contacts the company again next month, about something entirely unrelated to their earlier payment issue. Without memory, the agent starts from absolutely nothing — no awareness that this customer had a card- related payment problem last time, no awareness they’d stated a preference for email contact over phone. Every interaction becomes a fresh start, regardless of how much was learned about this specific customer previously.

Memory exists precisely to prevent that. It’s what lets an agent behave less like meeting a stranger every single time and more like picking up a relationship with real, accumulated context behind it. This matters more for some agents than others — a one-off, self- contained internal tool doesn’t need it, exactly the judgment call covered in Module 3 — but for anything with recurring users or recurring situations, memory is what turns isolated task completions into something that improves with accumulated experience.

The finer-grained distinctions worth knowing

Within memory specifically, a few more precise terms are worth having:

Working memory is the narrowest slice — specifically the information actively needed to complete the current step, a subset of the broader state rather than something separate from it. Short-term memory is scoped to the current session or task and discarded once it ends — functionally, this is what we’ve been calling state throughout this module, just under a different name you’ll see used interchangeably in some material.

Long-term memory is what persists across separate sessions — the customer’s stated seat preference, carried forward indefinitely. External memory is where that long-term memory lives — a database, a file, or (commonly) a vector store, the same retrieval mechanism you already studied in your RAG course.

We’re deliberately not re-deriving how vector retrieval works here; the connection worth holding onto is that retrieving relevant long-term memory for the current situation is the same problem your RAG course already solved for documents — embed the memory, embed the current context, retrieve what’s relevant, rather than dumping everything ever stored into every context.

A real, concrete illustration: how Claude Code handles exactly this problem

It’s worth seeing this three-way distinction show up as a, documented engineering mechanism in a real, widely-used product, because the problem it solves is precisely the one this module has been describing.

During a long coding session, Claude Code’s context — everything currently loaded: your instructions, every file it’s read, every tool result, its own prior responses — grows, and it will eventually approach the model’s context window limit. When that happens, Claude Code performs what it calls compaction: it summarizes the accumulated conversation history to make room for continued work.

And critically, Anthropic’s own documentation is specific about what that summarization deliberately tries to preserve versus discard: “Claude Code intelligently retains the information that matters, including what files you’re working on, the decisions you’ve made, and the state of your current task, while discarding the back-and-forth that led you there.” ([Developing with AI Tools, Compacting Claude Code Sessions](https://m. academy/lessons/compact-conversations-claude-code/))

Read that sentence again slowly, because it’s describing exactly the distinction this module is built around, in a real, shipped system. The raw, moment-to-moment context — every individual tool call, every intermediate exchange — is exactly what gets discarded. The state — which files matter, what’s already been decided, where the task currently stands — is exactly what gets deliberately extracted and carried forward, even as the raw context that originally contained it gets compressed away.

And engineers working with Claude Code on long sessions are explicitly advised to write anything that needs to survive even compaction itself — a subtle decision, a specific constraint — into a persistent file rather than trusting it’ll survive in the summarized state alone. That’s memory, in exactly this module’s sense: something deliberately promoted beyond the task’s own transient state into something durable.

It’s worth noting a second, distinct real example specifically for the memory side of this distinction, separate from state: several major consumer AI products now offer a persistent memory feature that explicitly operates across separate conversations — remembering a user’s stated preferences or facts from one session and making them available in a completely different, later session.

This is a, different mechanism from anything Claude Code’s compaction is doing — compaction manages one long-running task’s state as it grows; a cross-conversation memory feature is deliberately carrying specific, selected facts past the boundary of any single task entirely. Both are real, and they’re solving different problems, which is exactly why this module insisted on keeping the terms separate rather than treating “the agent remembers things” as one undifferentiated capability.

Applying all three to our recurring support agent

The flight-booking example above is useful precisely because it’s simple enough to see the three concepts cleanly separated. It’s worth now applying the exact same three-way lens to the support agent you’ve been following since Module 1, because a real production agent rarely deals with these concepts in isolation — they interact, and seeing that interaction matters.

Picture the agent mid-investigation, at the exact moment from Module 4 where it’s just discovered the payment-history mismatch and is about to decide whether to check the gateway. Its state at that instant looks something like:

customer_status: active, good standing
payment_history: 2 declines, reason "insufficient_funds"
customer_claim: "gateway issue"
gateway_status: not yet checked

The context assembled for this specific next decision doesn’t necessarily include all of that state verbatim — it includes whatever subset is relevant to deciding the next action, plus anything pulled from memory that’s applicable, plus the tool description for check_payment_gateway.

If this agent also has access to long-term memory about this customer — say, a note from three months ago that this same customer previously disputed a charge that turned out to be a duplicate authorization — that specific memory would be worth retrieving into this step’s context, because it directly informs how skeptically the agent should treat the current “insufficient funds” result.

A memory about, say, this customer’s preferred contact language would not be relevant to this particular decision, even though it’s stored in the same memory system — which is exactly the selective-retrieval discipline covered above, now made concrete: memory should be retrieved based on relevance to the current step, not included wholesale just because it exists.

Notice what happens once this ticket resolves. The state — every field in that structure above — is discarded entirely; its job is finished. But if something from this specific investigation is worth remembering for next time — perhaps that this customer’s card does have recurring insufficient-funds issues, worth knowing before jumping to a gateway-fault assumption in a future ticket — that specific, selected fact would need to be deliberately written to memory before the state disappears.

This is worth stating plainly, because it’s a, easy-to-miss engineering requirement: nothing automatically promotes state into memory. If a fact worth keeping isn’t explicitly written somewhere durable before the task ends, it’s simply gone, the same way Claude Code’s compaction discussion above warned that a subtle decision not explicitly saved to a file won’t reliably survive a compaction event either. The mechanism is different; the underlying lesson — durability requires a deliberate act, not an assumption — is exactly the same one.

The real risk of getting context assembly wrong

It’s worth being explicit about one more failure mode specific to context, distinct from a state or memory problem: even when state and memory are both being tracked correctly, a context that includes too much — every field of accumulated state, every marginally-related memory, the full text of every prior tool result — can degrade a model’s ability to reason well, simply by burying what matters under what doesn’t.

This is directly the same lesson your RAG course already taught about retrieval: more retrieved content isn’t automatically better, because irrelevant content competes for the model’s attention with content that’s relevant. Context assembly deserves the same deliberate selectivity RAG retrieval does — treating “what should this specific step see” as a design decision, not a matter of including everything available just because it’s technically accessible.

Real memory systems: how much space, what kind, and what went wrong

Everything above has been principle. It’s worth grounding it in four real systems, ordered deliberately from the roughest, earliest approach to the most architecturally rigorous one, because the progression itself teaches something the individual examples don’t: memory got more disciplined over time specifically because early, looser approaches ran into real, predictable problems.

**The early, rough precedent: AutoGPT-era vector memory. ** The first wave of autonomous agents, referenced repeatedly throughout this course, mostly implemented long-term memory the same simple way: embed everything the agent learns, store it in a vector database (commonly something like Pinecone, Redis, or Chroma at the time), and retrieve semantically similar entries when needed. This is the same retrieval mechanism your RAG course already covered, applied to an agent’s own accumulated experience instead of a curated document set.

The kind of memory here was almost entirely long-term, external memory — nothing resembling the tiered structure you’re about to see below.

The challenge this ran into was close to exactly what this module warned about in the section above: without deliberate selectivity, retrieval could pull back memories that were technically similar but not relevant to the current situation, quietly bloating context and competing for the model’s attention with information that mattered — a direct, real-world instance of “more retrieved content isn’t automatically better,” now happening to an agent’s own memory instead of a document store.

There was no widely-adopted resolution within these early systems themselves; the resolution came from the field building more disciplined approaches afterward, three of which follow.

**The deliberately small, capped approach: ChatGPT’s Saved Memories. ** OpenAI’s consumer memory feature draws a hard line between two different mechanisms, and the reasoning behind that split is directly instructive. “Saved memories” — specific facts ChatGPT decides are worth keeping, or that a user explicitly asks it to remember — are kept to a small, curated list, commonly cited around roughly 1,200 to 1,400 words total across all saved entries combined.

That’s a tight cap, and it’s a deliberate design choice, not a technical limitation: every single saved memory gets fed into the model as context on every conversation, which costs real tokens and competes for the model’s attention on every single chat, whether or not that particular memory is relevant to what’s being discussed right now. ([MemX, ChatGPT Memory Is Full](https://memx.

app/blog/chatgpt-memory-full-what-to-do/)) The challenge here is precisely the context-assembly risk covered earlier in this module, playing out at real, planet-scale usage: keep this always-injected memory too large, and every conversation gets measurably slower and more expensive, regardless of whether that conversation has anything to do with most of what’s stored.

OpenAI’s resolution is a two-tier design, echoing the state-versus- memory split from earlier in this module: a small, always-present “Saved Memories” list, plus a separate, much larger “reference chat history” that isn’t injected into every context by default but is instead searched on demand — a real production instance of exactly the selective-retrieval discipline this module has been arguing for throughout.

**The compaction approach: Claude Code, revisited with real numbers. ** You already saw this system’s basic mechanism earlier in this module — raw context discarded, state deliberately preserved, durable facts explicitly promoted to files.

Worth adding the concrete scale this challenge operates at: Claude’s models run with context windows as large as 200,000 tokens, and some configurations extend considerably further — enormous compared to ChatGPT’s roughly-1,300-word saved-memory cap, precisely because Claude Code’s job (holding an entire coding session’s file contents, tool outputs, and reasoning trace) is a fundamentally larger memory problem than consumer chat personalization.

Even at that much larger scale, the same challenge from the AutoGPT example still applies in a different form: a long enough session will still eventually exceed even a 200,000-token budget, and repeatedly re-reading whole files or receiving verbose command output can make a session grow large surprisingly quickly.

The resolution — compaction, deliberately preserving state (files, decisions, task progress) while discarding raw back-and-forth — is a more sophisticated answer than ChatGPT’s simple hard cap, precisely because a coding task’s state is much harder to compress down to a fixed word count without losing something that in reality matters.

**The most architecturally rigorous approach: MemGPT and Letta. ** This is worth understanding in real detail, because it’s the closest thing to a formal, published answer to the exact problem the three examples above were each solving in their own more ad-hoc way. The MemGPT research paper — “MemGPT: Towards LLMs as Operating Systems,” Packer et al.

, 2023 — proposes treating an LLM’s limited context window the same way an operating system treats limited physical RAM: as a scarce resource that needs active, deliberate management rather than passive accumulation. ([MemGPT paper, arXiv](https://arxiv. org/pdf/2310. 08560)) Concretely, this means a two-tier architecture: main context — the tokens visible to the model right now, analogous to RAM — and external context — everything else, analogous to disk storage, invisible to the model until it’s deliberately paged in.

The system that grew out of this paper, Letta, formalizes this further into three named tiers: core memory, small editable blocks that live permanently in context (the “human” block tracking facts about the user, the “persona” block defining the agent’s own role); recall memory, searchable conversation history stored outside the immediate context; and archival memory, longer-term storage the agent queries explicitly via tool calls. ([Vectorize, Mem0 vs Letta](https://vectorize. io/articles/mem0-vs-letta))

What sets this apart from the other three examples is who decides what moves between tiers. In ChatGPT’s system, memory promotion happens largely automatically, behind the scenes. In Claude Code, compaction is a system-triggered event, with the user advised to manually promote critical facts to files beforehand.

In MemGPT and Letta, the agent itself is given explicit tool calls — core_memory_append, archival_memory_search, and similar — letting it decide, as part of its own reasoning, what’s worth keeping in its small, always-visible core memory versus what should be pushed out to searchable, external storage.

This is the most self-directed of the four approaches, and it comes with a real, honest trade-off worth naming: memory quality now depends directly on the model’s own judgment about what’s worth retaining — if the model fails to explicitly save something before it would otherwise be lost, it’s simply gone, the same durability requirement covered earlier in this module, now placed even more squarely on the model’s own reasoning rather than on a separate, guaranteed system process.

Numbered Walkthrough: Where Each Piece Lives

Consider a travel agent helping Ravi book a flight.

1. Long-term memory store: seat_preference=aisle was saved from an earlier trip, with Ravi’s permission.
2. Task state: destination=Delhi, date=12 October, selected_flight=none, and payment_status=not_started belong to this booking.
3. New observation: a search tool returns 20 flights.
4. Context assembly: the application sends the model the goal, relevant task state, Ravi’s aisle preference, and the best five flights—not every stored conversation and all 20 results.
5. Decision: the model recommends flight AI-805 and explains why.
6. State update: selected_flight=AI-805; permanent memory is unchanged unless Ravi asks the system to remember something new.

The database may hold memory and state, but information enters the model’s reasoning only when the application places it in the current context.

Common Misconception

Incorrect idea: If information is stored in memory, the model automatically knows it.

Why it is incorrect: Stored information must be retrieved and included in the model’s context. Poor retrieval can make useful memory effectively invisible, while retrieving too much can crowd out important facts.

Key Takeaways

• State is an agent’s accumulated, in-progress understanding of one specific task — it starts empty, fills in as the task proceeds, and is discarded once the task ends.
• Context is everything handed to the model for one specific reasoning step — assembled fresh each time from state, relevant memory, and the latest observation, not stored as its own persistent object.
• Memory is information deliberately retained beyond a single task, available to inform future, separate interactions.
• State exists because multi-step tasks depend on later decisions being informed by earlier discoveries — you already watched this happen in Module 4’s loop walkthrough, where iteration 3 depended on what iteration 2 had already established.
• Memory exists because agents with recurring users or recurring situations benefit from not starting from nothing every single time — turning isolated task completions into something informed by real, accumulated experience.
• Working memory, short-term memory, long-term memory, and external memory are finer distinctions within this same framework — short- term memory is functionally the same thing as state under a different name; external memory is where long-term memory lives, commonly via the same retrieval mechanism your RAG course already covered.
• Claude Code’s context compaction is a real, documented illustration of this entire distinction in one shipped system: raw context gets discarded, state (files, decisions, task progress) gets deliberately preserved, and anything that needs to survive even further gets explicitly promoted to real, persistent memory.

Think Like an AI Engineer

• Go back to the flight-booking state structure at the start of this module. If the customer abandons the booking halfway through and contacts support again a week later to try again, which of those seven fields — if any — should carry over as memory, and which should start fresh as new state? Justify your answer for each field individually.

• Design what should happen when a long-running research agent’s context is about to exceed its window. Using this module’s vocabulary precisely, what should be discarded, what should be preserved as state, and is there anything in a typical research task that would be worth promoting to real memory rather than just preserved state?

• A teammate says “our agent has a bug — it keeps forgetting things mid-task.” Before touching any code, what specific question would you ask them to determine whether this is a state problem, a context-assembly problem, or a memory problem? Why does the answer change what you’d go fix?

• Think of a real, recurring interaction from your own domain — a returning user, a repeated kind of request. What’s the smallest, most useful piece of memory you could persist about them that would meaningfully improve a future interaction, without drifting into storing more than matters?

Module 8 takes everything from Modules 1 through 7 and asks a practical question: how much of an agent’s behavior should be left to dynamic decision-making, versus fixed in advance as a predictable workflow? We’ll walk the full spectrum from a rigid, fixed workflow all the way to a fully autonomous agent, with a real, concrete answer for when each point on that spectrum is the right engineering choice.