Notes on Memory Architectures for Conversational Agents

Most conversational agents in production today operate without any persistent learning from their own outcomes. Each session begins from a blank state. The model may be extraordinarily capable in the moment, but the system around it has no memory of what worked the last time a similar situation appeared, no record of which suggestions were accepted and which were silently ignored, and no way to improve from experience the way a human collaborator would. This limitation is not a bug in any particular agent. It is a consequence of the fact that the field has not yet settled on what a memory system for an intelligent agent should actually look like.

We have been sitting with this question for some time, and the framing that has proven most useful so far comes from an unexpected direction: the structure of human memory as it is currently understood in cognitive science. The analogy is not perfect — no analogy between biological and artificial systems ever is — but it has given us a vocabulary and a set of design constraints that we did not have before.

Three kinds of memory, not one

The first observation is that human memory is not a single thing. It is at least three distinct systems working in parallel. There is working memory, which holds a small amount of information for a short time while reasoning is happening. There is episodic memory, which stores specific past events as reconstructable scenes. And there is what we might loosely call procedural or intuitive memory, which is what remains after many episodes have been consolidated into generalized patterns, heuristics, and reflexes. A skilled practitioner does not remember every individual case they have handled; they remember the shape of the class of cases, and they can recall specific episodes when the situation demands it.

Most current agent memory systems collapse these into one layer. They store transcripts, or embeddings of transcripts, and retrieve them by similarity. This is useful, but it is analogous to asking a human to function using only their episodic memory, with no consolidation into intuition and no working-memory layer to coordinate reasoning in the moment. The retrieval is correct in a mechanical sense and yet the behavior that emerges from it is oddly memoryless, because the system has no mechanism for letting past episodes actually shape present judgment.

The role of consolidation

The second observation, and the one we keep returning to, is that the most interesting thing human memory does is not storage. It is consolidation. During rest, and especially during sleep, the brain appears to replay recent episodes and slowly integrate the useful patterns into longer-term structures, while allowing the specific details of most episodes to fade. The result is that a person wakes up slightly better at the things they practiced yesterday without needing to consciously remember any particular practice session.

An agent that wants to learn from its own work at anything like this level of fluency probably needs an analogous process. Not literally sleep, but a separate, asynchronous phase in which recent episodes are reviewed, outcomes are examined, patterns are extracted, and the operational state of the agent is updated to reflect what was learned. The interesting design questions here are not primarily about storage formats. They are about which events deserve consolidation, what is distilled from them, how the distilled knowledge interacts with everything the agent already “knows,” and how to prevent the whole process from drifting in directions that make the agent worse over time.

What counts as an outcome

A third observation is quieter but has turned out to matter more than we expected. In order for an agent to learn from experience, it has to be able to tell what experience actually taught. For humans, this information often comes from the environment: the stove is hot, the sentence was well received, the code compiled, the patient recovered. For an agent operating at a distance from physical consequences, the signal has to come from somewhere else — from explicit user feedback, from downstream success metrics, from internal consistency checks, or from some combination of the three. The design of the outcome signal turns out to be as important as the design of the memory itself. A memory system that faithfully records the wrong signal will confidently learn the wrong lessons.

Open questions

We are not yet in a position to claim a complete answer. Several things remain genuinely open. The right granularity of an episode is unclear: is it a single turn, a full session, a task, or something else? The tradeoff between recall precision and consolidation generality is not obvious and probably depends on the domain. The mechanisms for forgetting — deciding what to let go of, and when — are at least as subtle as the mechanisms for remembering. And the relationship between this kind of long-horizon memory and the short-horizon context window of the underlying model is an area where the right interface has not yet been designed.

What we can say is that the problem is real, that framing it in terms of consolidation rather than retrieval has been clarifying, and that the work on these questions is ongoing. We expect to return to this subject in future notes as our experiments give us something more concrete to share.