The Shape of Forgotten Knowledge
If you ask someone to explain a book they read years ago they won’t recite passages. They’ll give you a blurry, compressed summary. The details are gone but the shape remains. The themes, the key moments, maybe a striking sentence or two but not the exact words. What we remember is not the book itself but an abstraction of its ideas filtered through our own understanding.
Large language models work the same way.
There’s a common misconception that AI models like GPT “store” the internet. That they hoard articles, books and Wikipedia pages like a vast digital library. But if that were true, they would be able to recall things perfectly. Instead they often get facts wrong, forget niche details or worse, confidently hallucinate something that never existed. This isn’t because they’re unreliable in the way a faulty database might be. It’s because they don’t store knowledge at all. They compress it.
Compression as a Necessity
All knowledge is compressed because raw storage is impossible. Even for humans. If we remembered every moment of our lives in perfect detail, our minds would be overwhelmed. Instead, we extract meaning. We generalize. We forget what doesn’t seem important. The result is that we retain concepts and relationships rather than exact details.
LLMs do something similar. They don’t memorize facts. They recognize statistical patterns in language. If a model “knows” that Paris is the capital of France it’s not because it has a hardcoded fact stored somewhere it’s because in the vast corpus of text it has seen, “Paris” and “capital of France” appear together often enough to form a reliable pattern. It’s a probability distribution not a knowledge database.
This means LLMs are lossy by nature. They strip away specifics and retain the structure. They are in effect, an approximation of human knowledge sometimes insightful, sometimes eerily accurate, but never perfectly faithful.
Where This Compression Fails
The same way people misremember, LLMs hallucinate. They aren’t retrieving facts, they are reconstructing knowledge from their compressed patterns. This works well for common information but breaks down in areas where specificity matters.
Ask a model about a famous historical event and it might get most things right because those patterns are well represented in its training data. But ask it to generate citations for an academic paper and it might confidently fabricate sources because it doesn’t actually “know” any citations. It only knows what a citation should look like.
The problem isn’t just that LLMs lose information, but that they don’t know what they don’t know. If a human forgets something they might admit it. An LLM on the other hand will fill in the gaps, generating something plausible rather than leaving a void. This is both its greatest strength and its biggest flaw.
Why This Matters
Most people still think of AI like a search engine: you ask a question and it retrieves an answer. But that’s not what’s happening. LLMs aren’t retrieving, they’re predicting. They’re constructing responses dynamically based on compressed knowledge, not looking up information from a database.
This has real implications. It means we shouldn’t rely on LLMs for precise facts without external verification. It also explains why they can be so useful for reasoning, brainstorming and synthesis because they work like humans do, approximating meaning rather than storing data.
The Future of Knowledge Recall
Can AI compress less destructively? Could a model blend its generative capabilities with retrieval mechanisms, allowing it to verify details in real time rather than hallucinating them?
Some approaches already attempt this like retrieval augmented generation (RAG), where models search external sources before responding. But the deeper question remains: What does it mean to “know” something? Whether for humans or AI, knowledge isn’t about storing raw data it’s about forming connections, recognizing patterns and making sense of what remains after we forget the rest.