AI + Memory for Beginners
In my last post, I outlined why I believe that memory is all that matters.
The AI system that has access to your digital memories will be the most useful.
You will use that AI system the most because it is the most useful.
This use will build more memory for that AI system.
And this additional memory will make it even more useful.
It’s an unbeatable flywheel.
And this is why memory is all that matters.
But what is digital memory exactly, and how does it work in AI systems?
Digital Memory is made up of the following 4 components:
Here is an overview of how each works in practice:
1. In-Weight Memory
If you ask ChatGPT what’s the capital of North Carolina, it will tell you Raleigh.
This is because the answer is in the training data the AI model that powers ChatGPT was trained on.
Once an AI model has been trained, its in-weight memory is fixed. Adding new in-weight memories requires additional training.
2. In-Context Memory
I used to have a tab in ChatGPT that I used as a food diary.
If I asked it what I ate last Friday it would tell me.
The AI was not trained on this data so it’s not pulling this memory from its in-weight memory.
It answered accurately because the answer is in the same chat window where I asked the question.
The answer is in the in-context memory of the system.
3. Retrieval-Based Memory
Let’s say you ask ChatGPT for the score of last night’s Yankees game.
That information isn’t stored in the model’s training data (in-weight memory), and you haven’t mentioned it in the current conversation (in-context memory).
So how does it answer?
In this case, ChatGPT retrieves the answer from the internet. It’s not remembering, it’s retrieving the memory from an external source in real time.
This is what we mean by retrieval-based memory: the ability of an AI system to fetch information from outside sources like the web, databases, documents or APIs.
Retrieval-based memory is not limited to public data. You can also use it with your own private data as well.
And this is where AI starts to get very personalized, and even more powerful. For example:
- Your health tracker data → Ask questions about trends, risks, improvements.
- Your email archive → Summarize conversations or surface forgotten tasks.
- Your financial statements → automate budgeting and make recommendations for spending and investment optimizations.
- Your text messages → Ask for relationship advice based on past conversations.
- Your child’s report cards → Ask for personalized tutoring.
The possibilities for using AI to generate personalized advice by connecting it to our private data are endless.
4. Persistent-User Memory
The most powerful (and complex) form of AI memory is persistent-user memory.
Unlike in-context memory, which disappears when the conversation ends, persistent memory sticks around across conversations.
It allows the AI to build a lasting understanding of who you are, what you care about, and how to better assist you.
Let’s go back to the food diary example.
If I keep a food diary in one ChatGPT tab, and ask it what I ate yesterday in the same tab, it will give me the right answer.
This is because the answer is in the same chat window where I asked the question.
However, if I open a new tab in ChatGPT and ask the same question, it will not know the answer.
This is because ChatGPT does not have persistent-user based memory.
Or at least it didn’t until recently.
ChatGPT has now started rolling out persistent-user memory.
It can sometimes remember details from past chats, even across sessions.
That’s a big shift.
Building persistent-user memory isn’t easy. The AI system needs to:
- Decide what to remember and what to ignore
- Know when a memory is outdated and should be revised or discarded
- Handle conflicting or updated information correctly
And it needs to apply its memory proactively.
If I’m a vegetarian and I ask for restaurant recommendations, it should suggest only vegetarian restaurants, without me reminding it every time.
And if I change my mind later and start eating meat, I should only have to tell my AI system once. It should then incorporate this new memory into all future recommendations.
This kind of seamless, adaptive behavior is what makes persistent memory so powerful.
Storing, retrieving, and updating memories in a similar way to your physical mind.
But much more powerfully.
Once persistent user memory is perfected it will never forget any information you give it.
It will be available to provide you with 100% personalized advice on any topic 24/7/365.
And it will be smarter than the smartest experts on most of those topics.
In other words we will begin to rely on the system that holds our persistent-user memory for everything.
As our personal:
- Dr.
- Lawyer
- Therapist
- Career Coach
- Financial Advisor
- Kids tutor
- The list goes on and on.
Once an AI system owns your user based memory, moving to another AI system will be difficult or impossible.
It will require exporting all the conversation history since you started using the system. Assuming the system allows you to do this, that's easy.
But what's not easy is recreating all the:
- Connections to external data sources you are using for your retrieval based memory.
- Connections the AI system has made between your digital memories over years.
- Workflows and AI agents you rely on to accomplish tasks using your digital memories in that system.
In short, once you come to rely on an AI system that holds your memory, switching systems will be like the digital equivalent of brain surgery.
You can do it, but will you want to?
And this is why the most important question of our time is not who will have the most powerful AI algorithm.
But who will own our memory?
If we continue using ChatGPT or other Big Tech AI systems we will not own our memory.
Big Tech companies will.
You’ll rely on ChatGPT or another Big Tech AI system for all areas of your life.
They will decide how the AI system you use works.
What you can and cannot do with it.
It will have the values that the Big Tech company that owns it says it should have, which may be very different from yours.
There will be one answer to questions like “is there a god” and “who is the best candidate for president". And that answer will be whatever the Big Tech company that owns your AI system says it is.
And once you are locked into their system it will be difficult or impossible to move.
But it doesn’t have to be this way.
The technology exists today for each of us to build, own, and control our own AI systems.
So we can decide how we use the AI system we rely on.
So we are free to decide for ourselves what we will and will not do with that system.
To choose or build the AI algorithms that best represent our values.
To be able to swap in and out different components of that system as we please.
And to build a persistent-user memory that we own, control, and benefit from, instead of some Big Tech company.
Thanks for reading!
Dave
P.S. If you’re new here I’m the co-creator of BrainDrive.ai an ecosystem designed to help power the user owned AI movement. If you’re interested in building, controlling, and benefiting from your own AI system, join us here.
