I was the only memory my AI agents had. So I built them a shared one

Every new session started from zero, and I kept re-explaining what we had decided and why. Here’s what changed when all my agents began reading and writing the same memory, and what it still can’t do.

Every new session with an AI agent started the same way: me, explaining again.

What the project is. What we decided last week. Why we picked this option and dropped the other one. Where exactly we stopped.

And a few messages later: “Hold on, don’t suggest that. We already tried it.”

I live in Leipzig, Germany, and I didn’t write code before January 2026. Today AI agents do the building: my sites, my servers, my social media posts. I set the direction and review the result. It works surprisingly well. Until the session ends.

I had become the memory

In July I split the work between agents. Claude Code took the sites and servers. Codex and ChatGPT took social media.

The split made sense. The memory didn’t. Some of these tools have a memory of their own, but it stays inside that one tool. So I kept walking between them: copying, pasting, retelling, reminding.

The one holding the whole picture was me. Why we made a decision. Why we didn’t go the other way. What was promised, and to whom.

The “why” turned out to matter most. A week later, the option we rejected can look like a great idea again, if nobody remembers what was wrong with it.

And old information can be worse than none. In October it bit me properly: some agreements got lost because an old status was taken for the current one.

So I built Librarian: one memory for all my agents.

One memory, four simple rules

Under the hood it’s a database my agents connect to through MCP, the open protocol AI tools use to plug into outside data. Today Claude Code, Codex and ChatGPT all read from it and write to it.

What makes it useful isn’t the database, though. It’s four rules.

One memory that every agent reads and writes. Agents save things as they work, not at the end: a decision and the reason behind it, a promise with a date, who I talked to and what we agreed, the result of a test, including the approaches we dropped and why. Before saving, they search, so the same fact doesn’t get written twice. When a piece of work ends or changes hands, the agent leaves a handoff: what’s done, what’s decided, what not to do, what’s next. Agents are told never to save passwords, keys or whole chat logs. On top of that, when a terminal session ends, a short summary lands in memory automatically: what we worked on, the problem, how it was solved, what’s still open.

Every record has a source and a date. A record says where it came from (a chat, a code repository, a document, a Telegram message) and when it was written. If there’s no link to point to, the record says so instead of making one up. My own words are saved as confirmed. An agent’s conclusion is saved as provisional, so a guess never passes for a decision.

Old records get marked “possibly stale.” Some things are temporary by nature: progress notes, next steps, open problems. Once one of those is older than 30 days, the agent gets it with a flag: possibly stale, check before acting. Decisions and constraints don’t expire with age. They stay valid until something replaces them. Search also leans slightly toward newer records.

A correction replaces the old record instead of piling up next to it. When something changes, the agent doesn’t add a second note that quietly contradicts the first. It files a correction. The old record stays in the history, marked as replaced, and drops out of normal search. The correction shows up right where the old record used to be.

Here’s how that plays out. This is a made-up example, not a record from my memory. In September an agent saves: “The newsletter goes out on Tuesdays.” Type: decision. Source: chat. In October I change my mind, and an agent files a correction: “Thursdays.” From then on, any agent that asks about the newsletter sees Thursday. Tuesday is still in the history, marked as replaced. And a September next step like “draft the welcome email” comes back flagged as possibly stale, so the agent checks whether it’s still open instead of doing it twice.

What changed in my day

Now I open a session and write something like: “Pull up the context on this project. Where did we stop?”

The agent finds the decisions with their reasons, the open questions and the next step. It sees when each one was saved and whether it may be out of date. I don’t retell the conversation.

Switching tools no longer resets anything. I can talk something through in ChatGPT and pick it up in Claude Code from the same point. If I change tools one day, the memory stays where it is. That was deliberate: I don’t want any single tool to become impossible to replace.

For me, the most useful part is coming back to a thought I’ve already worked through properly. I can put a project aside, work on other things, open it again later and land where I left off.

Personal things are kept apart. Work projects are the default. Personal, family, health and money topics sit in separate areas, and an agent only touches them when I explicitly ask.

What it isn’t (yet)

It isn’t magic. Whatever quality goes in is the quality that comes out. The automatic summaries are written by a model, and a model can miss or distort details. So I still double-check answers before anything important.

“Possibly stale” is about age, not truth. A fresh record can still be wrong. And a contradiction only gets resolved when someone files the correction.

It isn’t fully offline either. The memory database runs on a Raspberry Pi I own. But the session summaries are written by a cloud model (Google’s Gemini), and the assistants that read the memory are cloud services too. If you work with confidential client data, decide what goes in before anything does.

What I’m working toward:

• flagging contradictions automatically, when a newer record disagrees with an older one and nobody filed a correction;
• showing the source and date next to every fact in every answer, not only inside the record;
• a simple web page where I can browse, fix and archive records myself. It’s built, but not switched on yet;
• a free edition. I’m preparing it; it isn’t out.

If you keep re-explaining things too

If every new chat starts with you retelling the same context, the question is less “which AI?” and more “where does the memory live?”

I set this up for others as well, for one person or for a whole team. It starts with a 2-minute check: a few questions about where your knowledge lives today and which AI tools you already use. I read every answer myself and reply by email with a short memory map: what to connect first and where you’d ask your questions.

Want the same memory for yourself or your team? Take the 2-minute check: brain.sl-sp.de/setup

---

Slava Spitsyn is an entrepreneur in Leipzig, Germany. He builds his company with AI agents and sets up the same systems for others.

---

← All notes