The model sees your whole conversation for the first time on every turn

The exchange that feels like memory

You tell a chatbot, three messages into a conversation, that your dog is named Biscuit. Several messages later you ask for a dinner idea, and it suggests a recipe and adds a note about keeping the onions away from Biscuit. It reached back to something you said and used it. The natural reading is that the chatbot is holding the conversation in memory, the way a person keeps the thread of a talk in their head.

That reading is wrong in a specific and useful way. The model keeps nothing from one turn to the next. The name Biscuit is available later for one reason: the application typed your whole conversation, that earlier message included, back into the model as fresh input. On each turn two things move. The application hands the model an input, and the model hands back a reply. Everything the chatbot appears to know about the conversation sits inside that input, assembled again from scratch, and nothing is kept between one reply and the next request.

This is not a quirk of one product. Anthropic says it plainly in its own API reference: “The Messages API is stateless, which means that you always send the full conversational history to the API.”¹ Stateless is the settled fact everything else rests on.

Where the earlier words come from

If the model retains nothing, then the earlier turns have to arrive on the later request some other way. They do: the application resends the entire running transcript as the input every turn. The literal text of every prior message is placed in the input again, ahead of your new one, not referenced by an identifier the model looks up.

Anthropic’s own documentation shows the growth in a single example. Send one user message and the request reports 12 input tokens. Continue the conversation to three messages, a user turn, the model’s reply, and your next user turn, and the following request reports 30 input tokens.¹ The count rose because the earlier turns are physically back in the input, counted again, not because the model remembered them.

The other major API owner describes the same shape. OpenAI states that each text-generation request is independent and stateless, and that a multi-turn conversation is something the developer builds by passing the prior messages as parameters on every request.² The endpoint that returns the reply generates it from the list of messages that make up the conversation, supplied each call.³ This is settled engineering, stated by the parties that own the interface.

One everyday behavior drops out at once. A brand-new chat starts blank because there is no prior transcript to resend: the model is handed only your first message, with nothing earlier to reach for.

Why one call keeps nothing from the last

Resending the whole transcript looks wasteful, but the application has no choice. A single model call carries nothing out of the call before it, for two settled reasons.

The first is the architecture. The transformer, the network design every current chatbot is built on, is “based solely on attention mechanisms, dispensing with recurrence and convolutions entirely.”⁴ The word that matters is recurrence. Older designs passed a running hidden state from one step to the next, a place a memory of earlier input could live. The transformer removed it. A forward pass attends over the tokens it is handed and nothing else, so there is no slot in the design for a memory of a separate, earlier call.

The second is that the one piece of within-call state that does exist is deliberately thrown away. While the model writes a reply it keeps a key/value cache, so it does not recompute the earlier tokens for every new word it produces (this is the work of a single forward pass and its cache). That cache is per request. The serving system hands it out as the reply grows and returns it to a shared pool the moment the request finishes, so the memory is reused by whatever request comes next.⁵ An independent implementation reference describes the same cache, built to avoid recomputation within one generation.⁶ It is a speed trick inside one reply, not a store that survives to the next turn.

One more thing is worth separating out. A model’s weights freeze when its training run ends, and that freeze is about what the model knows in general. Statelessness is a narrower thing about the live conversation in front of you. Even mid-chat, with the weights fixed either way, one call retains nothing of the call before it. The frozen weights are why the model cannot learn your dog’s name into itself. Statelessness is why it has to be told the name again, in the input, on every single turn.

How a long chat loses its start

Once the whole transcript rides along on every turn, it only grows, and it grows inside a fixed space. That space is the context window, the amount of text a model can take in at once. On Anthropic’s models the window runs to 200,000 tokens on the baseline and up to 1,000,000 on newer ones, and the whole conversation plus the reply being written has to fit inside it.⁷ The window is working memory for this one conversation, and the documentation is careful to mark it as a different thing from the corpus the model was trained on.⁷

A long enough conversation, thousands of words of back-and-forth, eventually pushes against even a large window, and then something has to give. The window is a hard edge. On the Claude API, if the input alone is over the window, the request fails outright with a 400 error reading “prompt is too long.” If the reply instead runs into the ceiling as it is generated, the model stops, with the reason model_context_window_exceeded.⁷ An application cannot let the conversation grow without limit, so it has to decide what to drop or compress as the edge approaches. That decision belongs to the application, not the model.

What each product decides varies, and much of it is not published. Anthropic documents three of its own behaviors. A chat interface such as claude.ai can drop the oldest turns first, on a rolling basis. Server-side compaction can summarize older turns so the thread continues past the limit.⁷ And context editing can clear stale tool results out of the window as the token limit nears.⁸ What it does not publish is the exact point each of these triggers or the wording a summary uses, and other products do not always say what they do at all. So “the model forgot the start of our chat” resolves to something concrete but product-specific. The application dropped or summarized the oldest turns to keep the rest inside the window. Handed the trimmed transcript, the model has no way to miss what is no longer in front of it.

What a “memory” feature really stores

One behavior seems to break the rule. Some products recognize you across separate chats. Open a fresh conversation weeks later and the product already has your name, your preferences, a project from before, even though each chat on its own starts blank. If the model keeps nothing, how does anything survive between conversations?

It survives outside the model. A memory feature is the application storing facts somewhere of its own and pasting the relevant ones back into the input on later turns. Anthropic’s memory tool documents the mechanism firsthand: the model stores and retrieves information across conversations as files that persist between sessions, “building up knowledge over time without keeping everything in the context window.”⁹ On a later turn the application reads those files and places their contents into the input the model receives, as ordinary text, never into the model itself.

That last distinction separates a “memory” feature from memory in the ordinary sense. The window the facts are pasted into is working memory for the current conversation, marked in the same documentation as separate from the training corpus the weights hold.⁷ A saved fact lives in a file in the application’s own storage and reaches the model the same way your typing does, as text in the input, while the weights stay frozen. The model no more learns your name from a memory feature than it learns it from you saying it: in both cases the fact arrives as input and is gone the moment the reply is done.

The instruction tells the model plainly that it has no persistent memory, and that anything not saved to external storage is lost. What a product saves, and when it reads it back, is the product-specific part. That it arrives as input, not as anything the model learned, is the settled part.