How I Gave My AI Agents a Memory That Survives the Session

The Same Correction, Three Days in a Row

I corrected an agent about the same thing on a Monday, a Tuesday, and a Wednesday. Same correction. Same wording, near enough. Each time it accepted the correction, did the right thing, and finished the task well.

Then the session ended and the correction went with it.

That is the whole problem in one sentence. Agent sessions are amnesiac. The model holds everything you tell it for as long as the window lasts, and when the window closes the entire working relationship resets to zero. You are not training anything. You are re-briefing a competent stranger every morning.

For a while I treated this as the cost of doing business. Then I actually counted. The re-briefing was taking me the first ten or fifteen minutes of every session — context on the project, constraints I had already stated, preferences I had already expressed, decisions I had already made and did not want relitigated. Multiply by several sessions a day and the arithmetic stops being funny.

What I built to fix it is not clever. It is a folder of small text files and a set of rules about what goes in them. The value is almost entirely in the rules.

Why the Obvious Fixes Do Not Hold

The first instinct is to write a longer system prompt. Put everything in the instructions file, load it at the start of every session, done.

That works until it doesn't. A system prompt is a constitution, not a notebook. It holds things that are true always and everywhere — voice, jurisdiction, hard laws. The moment you start appending episodic facts to it, it bloats, and a bloated instruction file gets skimmed rather than obeyed. I have watched an agent follow rule one and rule forty and quietly ignore rules twelve through thirty. Not maliciously. There was just too much, and the important parts were buried under the incidental.

The second instinct is to dump whole conversation transcripts somewhere and have the agent read them back. This is worse. A transcript is mostly noise: false starts, abandoned approaches, things I said before I had thought them through. Feeding that back in makes the agent confidently wrong about decisions I later reversed. It remembers the argument, not the conclusion.

The third instinct is a vector database. Embed everything, retrieve the similar chunks. For large document corpora this is right. For a working memory of maybe eighty facts, it adds infrastructure, a failure mode, and an opaque retrieval step to a problem that plain text solves. I have a vault of documents that earns its search layer. The memory is not that.

What actually holds is narrower than all three: a small number of deliberately written facts, each in its own file, each explaining not just what is true but why it matters and how to act on it.

One Fact, One File

The unit of memory is a single file holding a single fact. Not a topic. Not a category. One fact.

This sounds fussy and it is the decision that makes the rest work. When a file holds one fact, updating it is unambiguous — the fact changed, rewrite the file. When a file holds six related facts, updating it means editing around five things you did not come to change, and over time such a file becomes a junk drawer that nobody reads to the bottom of.

Each file carries a small header: a slug for a name, a one-line description, and a type. The description is doing real work. When the agent is deciding what to pull into context, it reads descriptions, not bodies. A vague description means a relevant memory never gets recalled, which is the same as not having written it.

The body is the fact itself. For anything that is guidance rather than raw information, the body also carries two lines I insist on: why this is true, and how to apply it.

The Index Is the Whole Trick

Files on disk are inert. Something has to make the agent aware they exist without making it read all of them.

The answer is one index file, loaded into every session, holding exactly one line per memory: a title, a link, and a short hook. Nothing else. No content, no frontmatter, no summaries that drift out of sync with the thing they summarize.

The index is a table of contents, and the discipline is that it never becomes the book. Every time I have let content creep into the index — just this once, it's only two lines — I have ended up with two versions of the same fact disagreeing with each other, and the one in the index winning because it loads first.

Line, link, hook. The agent reads eighty hooks, decides three are relevant, opens three files. That is the entire retrieval mechanism, and it costs almost nothing.

What Must Never Go In

The rules about what to exclude have done more for this system than the rules about what to include. A memory store that accumulates indiscriminately becomes a swamp, and a swamp gets ignored.

**Anything the repository already records.** Code structure, past fixes, the shape of the build, what a function does. The repository is the source of truth for the repository. A memory that restates it is a second copy that will go stale, and a stale copy is worse than no copy because someone will trust it.

**Anything that only matters to this conversation.** The file I am editing right now. The bug I am mid-way through. These feel important because they are urgent, and urgency is not durability. If it will be meaningless in a week, it is context, not memory.

**Secrets.** Ever. Not in the memory, not in the shared vault, not in a file that is "only local." Local files get synced, backed up, and grepped by tools nobody remembers installing. If it is a credential, it lives in a credential store and the memory records where the credential store is.

**Relative dates.** "Last week" is a lie waiting to happen. Every date gets written absolutely, because the memory will be read months later by something that has no idea when it was written.

Memory Is Evidence, Not Instruction

There is a failure mode here that is worth being careful about, because it is subtle and it can do real damage.

A recalled memory arrives in the agent's context looking a lot like something you just said. It is text, it is authoritative in tone, and it is sitting right there next to your actual request. The temptation for the system is to treat it as a live instruction.

It is not. It is a note about what was true when someone wrote it. Those are different things, and conflating them means an agent obeys a preference you abandoned two months ago with the same force as a request you made ten seconds ago.

So recalled memories are marked as background context, and they carry no authority to override what is being asked now. If a memory says one thing and the person says another, the person wins, and the memory is a candidate for deletion.

The same skepticism applies to specifics. If a memory names a file, a function, a command-line flag — verify it still exists before acting on it. A memory is a strong hint about where to look, not a guarantee about what you will find. I have had an agent confidently invoke a flag that had been removed in a version bump, purely because a memory from before the bump said to. The memory was not wrong when written. It was just old, and nobody checked.

Deletion Is Maintenance, Not Failure

The instinct with a memory system is to treat every write as permanent and every deletion as loss. That instinct produces a store full of things that used to be true.

I delete memories regularly and it is the healthiest habit in the whole system. A preference I have changed. A project that shipped. A constraint that no longer binds. A fact that turned out to be wrong.

Before writing anything new I check whether a file already covers it. If one does, I update that file rather than creating a near-duplicate. Two memories that half-agree are worse than either alone, because whichever one gets recalled first shapes the answer and there is no way to tell which one that will be.

The store should stay small enough that I could read all of it in one sitting if I had to. The moment it grows past that, the index stops being scannable, the descriptions stop getting read carefully, and the whole thing degrades into the transcript dump I was trying to avoid. Small and current beats large and comprehensive, every time.

What Actually Changed

The obvious win is that I stopped repeating myself. That is real, and it is not the important part.

The important part is that corrections became cumulative. When I tell an agent it got something wrong and that correction is written down with its reason attached, the correction holds. The next session starts from a better position than the last one did. The relationship compounds instead of resetting.

There is a second effect I did not anticipate. Writing a memory forces me to decide whether something is actually a principle or just a mood. A surprising amount of what I was about to enshrine turned out, under the discipline of writing down *why*, to be a preference I could not justify. Those did not get written, and I think I am better off for having been made to notice.

The cost is honest to state: a few seconds of judgment at the end of a session, and a periodic cull. There is no heavy infrastructure, no service to keep running, no index to rebuild. Plain files in a folder and a single index line each. It could be moved to another machine with a copy command.

If you are building something similar, the part I would guard most carefully is not the format. It is the exclusion rules. Anyone can write facts down. The system only stays useful if you are ruthless about what does not go in, and honest about deleting what stopped being true.

Questions people actually ask

What is the difference between AI agent memory and a longer system prompt?

A system prompt holds what is always true — voice, hard constraints, laws that never bend. It is loaded in full every session, so it has to stay small enough to actually be obeyed. Memory holds episodic facts that are true for now and retrieved only when relevant. Mixing them bloats the prompt until the important rules get skimmed past. Keep the constitution separate from the notebook.

Why store agent memory as plain files instead of a vector database?

For a working memory of roughly a hundred deliberately written facts, a one-line-per-fact index is enough for the agent to decide what to open. A vector store adds infrastructure, a retrieval step you cannot inspect, and a new failure mode, in exchange for scale you do not need at that size. Large document corpora are a different problem and do earn a search layer. Start with files; add retrieval when files stop working, not before.

How do I stop an agent's memory from filling up with junk?

Exclusion rules, applied at write time. Nothing the repository already records, nothing that only matters to the current conversation, no secrets, no relative dates. Before writing, check for an existing file covering the same ground and update it instead of duplicating. Then delete memories that turn out to be wrong or have gone stale. The store should stay small enough to read in one sitting.

Should a memory that survives the session override what the user asks now?

No. A recalled memory is a note about what was true when it was written, not a live instruction. If it conflicts with what the person is asking now, the person wins and the memory becomes a candidate for deletion. Treat it as background evidence. If it names a file, function, or flag, verify that still exists before acting on it — a memory that was accurate when written can quietly go out of date.

What should actually go in a memory file?

One fact per file, with a short name, a one-line description used to judge relevance during recall, and a type. For anything that is guidance rather than raw information, add why it is true and how to apply it. The why is what lets the agent reason correctly in a situation the original rule never anticipated, instead of pattern-matching the wording and abandoning the rule the moment it conflicts with something else.

Someone asks me to remember something the code already says. What then?

Do not save it as-is — you get a duplicate that rots the first time the file moves. Ask what was non-obvious about it and save that instead. "The parser lives in that module" is something the code already states. "The parser lives there because the obvious location creates a circular import" is knowledge the code cannot state, and it is the part that would otherwise be rediscovered the hard way.