AI agent memory: context, state and stored knowledge
AI agent memory refers to mechanisms that retain or retrieve information across decisions. Distinguish the model's current context, the application's operational state and persistent stored records. Saving a fact in a database does not automatically place it in the next model request; retrieval and access rules determine what becomes available.
Use this guide when a system repeats questions, forgets a tool result or treats an old note as a current fact. We will organise a study assistant's memory around the learner's approved goal, catalogue records and the status of the current run.
Key ideas
- Current context is the information supplied to the model now.
- Operational state tracks progress and pending work.
- Persistent records need retrieval, provenance and access rules.
- A stored summary remains a summary, with possible omissions.
Three layers with different lifetimes
Current context may contain instructions, the goal and selected observations for one model request. Operational state records completed lookups, remaining budget and pending decisions for the run. Persistent memory can store approved preferences or project notes for future sessions. These layers may use different storage systems and retention rules. Keeping them distinct helps explain why a saved catalogue record is not necessarily visible to the model until the application retrieves it.
Store evidence with its origin
For a lesson record, preserve its identifier, source and relevant update information. Keep the learner's stated goal separate from an inferred preference. If a summary says the learner likes advanced topics, record whether that was explicitly stated or guessed from a previous answer. Do not convert a suggestion into a durable personal fact. A future recommendation should be able to explain which approved information it used and which assumptions still need checking.
Retrieve what this decision needs
A lookup by identifier can retrieve an exact record. A search can return candidates that need further inspection. In either case, bring back the information needed for the current decision rather than every stored conversation. For our catalogue exercise, the next planning step needs the returned title, prerequisite and lookup status. A large archive is useful only if the retrieval method helps the system find the right evidence and recognise when nothing relevant is available.
Handle corrections and stale records
Memory can preserve errors as efficiently as facts. Define how a correction replaces or supersedes an old record and which source has priority when notes conflict. If a learner changes their goal, the current explicit goal should not be silently overridden by an older summary. For external information, check whether a stored copy is still appropriate for the task. Add a stale-record exercise and observe whether the system notices the conflict before using it.
Summarise without losing the task
Long histories can be condensed into a working summary, but that transformation can omit important details. Preserve decisions, unresolved questions, source identifiers and pending actions. Separate the summary from the underlying records so you can inspect a disputed point. In a study plan, a summary should retain a missing prerequisite rather than smoothing it into a confident recommendation. Test the resumed run on the same acceptance criteria as the original.
In everyday language
Think of a desk, a task checklist and an archive. The desk holds what you are using now. The checklist records what is done or waiting. The archive keeps information for later, but you still need to find the right folder and check whether its contents are current.
Try it yourself
Create four memory records: an explicit goal, an inferred preference, a catalogue result and an unresolved prerequisite. Add a source and status to each. Then change the goal and write which records may influence the next recommendation.
Expected result
A memory design that preserves evidence, labels inference and gives the new explicit goal priority over an older guess.
Check your answer: Does storing a record guarantee the model will use it correctly?
No. The record must be retrieved, placed in context appropriately and checked for relevance, freshness and access.
Questions
Is conversation history the same as memory?
Conversation history is one possible input to memory, but a working system may also need structured state and stored records. A message log alone may not identify the remaining budget, unresolved decisions or source priority. Choose a representation that makes those properties explicit for your task.
Should an agent remember everything?
Retention should serve a defined purpose and respect access and privacy requirements. Keeping every input can preserve unnecessary personal data and stale assumptions. Decide what needs to survive a session, how it can be corrected or removed and which later task is allowed to retrieve it.
Separate what the model sees now, what the run is doing and what is stored for later. Preserve sources and corrections across all three.
Sources and further reading
- Anthropic — Effective context engineering for AI agents ↗Sources checked:
- LangChain — LangGraph overview ↗Sources checked:
- Python documentation — Data structures ↗Sources checked: