A model ends its reply by choosing a stop token like any other word

Two ways a reply can end

A reply that comes back through a serving system carries a note about why it stopped. The Anthropic Messages API returns a stop_reason field, and its value is end_turn when the model reached a natural stopping point and max_tokens when a requested length limit was hit.¹ Two codes, two different causes. One is a decision the model made. The other is a limit imposed on it from outside. The clean sentence you see and the word snapped in half you sometimes see are those two causes leaving their marks.

The stop is one more token the model samples

A model writes a reply one token at a time, a token being a chunk of text, usually a word or a piece of one. At each position it produces a probability for every token in its vocabulary, and one token is drawn from those probabilities and added to the reply.

The vocabulary includes a token that means the turn is over. Llama 3 writes it <|eot_id|>, an end-of-turn marker Meta's model documentation says the model emits once it "has determined that it has finished interacting with the user message."² It is an ordinary entry in the vocabulary. Meta and PyTorch's tokenizer gives it the integer id 128009, sitting in the same table as every word-piece.³ At each step the model puts some probability on 128009 exactly as it puts probability on "Paris" or "the." When the sampler draws that token, the decoding loop returns: the library that runs an open model stops as soon as the end-of-sequence token is produced.⁴ Emitting that token is the signal that ends generation.⁵ Stopping is not a separate part bolted onto generation. It is next-token prediction landing on the stop token.

Watch the probability of that token as an answer completes. Give the model "What is the capital of France?" and start it writing "The capital of France is." After "is," you would put nearly all your weight on "Paris" and almost none on the turn ending right there, because an answer word still has to come. After "Paris." the weight flips: nothing sensible remains to add, and the end-of-turn token becomes the likely next pick.

The trajectory is checkable, even if these numbers are illustration. Hugging Face's generation tools can print the probability the model assigned to any token at any step, so anyone can run this and read the real curve of the end-of-turn token's probability.⁴ Two levers in the same toolkit show the token is ordinary and not a hard rule. You can drive its probability to nothing so the model cannot stop yet, and you can force it at a length limit. A rule could not be switched off. A token can.⁴

Post-training taught the model where a turn ends

Where did the model learn to put high probability on the end-of-turn token at the close of a good answer? Not from grammar it worked out on its own. A model begins as a base model, trained to continue text by predicting the next token across a large pile of documents. A base model is not unable to stop. It emits an end-of-document token at a document boundary. Llama's base model produces <|end_of_text|>, which Meta says "is generated only by the base models,"² and Qwen's base model uses <|endoftext|>, a marker inserted between documents in its training data.⁶ What a base model lacks is the habit of ending at the close of a helpful answer, because in a stream of documents there is no turn to end.

Post-training put that habit in. A base model is fine-tuned for chat on data formatted as a sequence of messages, each turn wrapped in control tokens, and it learns to produce the end-of-turn token where a turn should close.⁷ Qwen sets its end-of-turn token <|im_end|> as the model's end-of-sequence token for chat, so producing it halts generation at a turn boundary.⁶ This is settled engineering, done on purpose, and the same across vendors.

The wider finding behind it is that size alone does not buy this. The InstructGPT work at OpenAI showed that making a base model bigger does not by itself make it follow a user's intent. Fine-tuning with human feedback is what aligns a model to that intent, and a much smaller fine-tuned model is preferred over the far larger base.⁸ Stopping at the end of a helpful answer is one of those trained-in behaviors.

The program that runs the model can stop it too

Everything so far is the model's own doing. The second cause is not. The model does not run itself. A serving program feeds it tokens and collects what it produces, and that program holds a limit on how many tokens it will accept back. In the Anthropic Messages API that limit is max_tokens, "the maximum number of tokens to generate before stopping."¹ When the count is reached, the program stops collecting and returns whatever it has, whether or not the end-of-turn token ever came. The reply is truncated. Anthropic reports this ending as max_tokens rather than end_turn.¹ Google's Gemini API reports the same event as a finishReason of MAX_TOKENS rather than STOP.⁹ The model did not stop itself. The program stopped it.

A caller can also hand the program a stop sequence, a string that forces a halt the moment the model produces it.¹ And these are not the whole list: the same Anthropic field can also report that the model refused, that it paused to use a tool, or that the context window filled.¹ Those endings are real but uncommon. The two you meet every day are the model's own end-of-turn token and the program's length cap.

The two endings leave different marks on the screen

You can usually tell which of the two ended a reply without reading any API field, because they leave different marks. The model's own ending lands on a boundary: a finished sentence, a closed code block, a completed list. It chose the end-of-turn token because nothing sensible remained to add, so the last thing on the screen is whole. A length cap has no such sense. It stops at a token count, wherever that count happens to fall, in the middle of "argu" with "ment" never written, or after the opening brace of a function whose body never comes.

A reply that reads "returns the sum of the two argu" was cut off by the cap, and reports max_tokens; the fix is to raise the limit and run it again, not to re-ask the model. A reply that reads "returns the sum of the two arguments." finished on its own and reports end_turn.¹ The first is the program's wall. The second is the model deciding it was done.