When ChatGPT Forgets: The Bizarre Memory Gaps That Make AI Feel Unhinged

Ever asked ChatGPT a follow‑up question and watched it completely lose track of the conversation? You’re not alone. It’s a quirk that many users find both amusing and unsettling, and some developers have even jokingly called it a kind of AI "psychosis."

The root of the problem isn’t a malfunctioning brain—because, of course, there isn’t one—but a hard limit on how much text the model can hold in its "working memory" at any given moment. This limit, known as the context window, caps the number of tokens (roughly words or word pieces) that the system can consider when generating a reply. For most versions of ChatGPT, that window sits somewhere between 4,000 and 8,000 tokens, which sounds like a lot until you realize a typical chat can easily surpass it after just a few back‑and‑forth exchanges.

When the window overflows, the model simply drops the oldest bits of the conversation. The result? It might ignore crucial details you mentioned earlier, repeat itself, or—worst of all—fabricate information that sounds plausible but has no basis in the prior text. This is what many refer to as "hallucination" and, in more colorful language, a form of AI psychosis.

Why does this matter beyond a momentary annoyance? In real‑world applications—think medical advice bots, legal assistants, or customer‑service agents—a lapse in memory can lead to misinformation, broken trust, or even harmful outcomes. Users expect continuity; they expect the system to remember the premise they set at the start of the conversation. When that expectation isn’t met, confidence erodes quickly.

Researchers are already tinkering with workarounds. One promising avenue is Retrieval‑Augmented Generation (RAG), where the model pulls in relevant documents or prior chat snippets from an external database before answering. Think of it as giving the AI a quick notebook to flip through, instead of relying solely on its short‑term recall.

Another line of development focuses on dynamic context windows that can expand on demand, or on "memory tokens" that flag important facts to keep in mind throughout the session. These ideas are still in the prototype phase, but they hint at a future where AI assistants can hold a conversation as seamlessly as a human would.

Until those breakthroughs become mainstream, the best you can do is keep reminders handy. Summarize key points every few turns, or feed the model a short recap when you sense it’s drifting. It’s a bit like nudging a forgetful friend—slightly repetitive, but it gets the job done.

So, the next time ChatGPT seems to lose its train of thought, remember: it’s not a glitch in its personality, just a limitation of its design. And somewhere in a lab, engineers are busy figuring out how to give it a better memory, one token at a time.