Chatbots overuse ‘delve,’ and the popular explanation for it failed its one direct test.

From a hunch to a counted spike

Dmitry Kobak and colleagues took 14 million English abstracts published in PubMed between 2010 and 2024 and, for each word, drew the line its frequency had been following before large language models arrived. Then they compared that projection to what 2024 actually looked like.¹ Most words stayed near their line. A small set jumped far above it, and the jump was abrupt, dated to the months after ChatGPT appeared in late 2022. The words that rose the most by ratio were low-frequency style words: forms of ‘delve,’ ‘underscore,’ and ‘showcase.’ From how much excess word usage they measured, the authors put a floor under the change. At least one in ten 2024 abstracts showed signs of being processed with a model, and as high as three in ten in some corners of the literature.¹ Treat that as a moving lower bound rather than a headcount. A later version of the same analysis revised it upward as more of 2024 came in.

The pattern is not an artifact of one database. Weixin Liang and colleagues ran a similar count on machine-learning conference peer reviews and found the same shape in a different genre. Particular adjectives that were rare in reviews from before ChatGPT turned common in the 2024 ones.²

These are counts and nothing more, a blunt tally of how often a word shows up. Two independent corpora moving the same way in the same year are hard to wave off.

Every word is a draw from a distribution

How do the words reach the page in the first place? A language model writes one token at a time, and each token it has already written becomes part of what it reads to choose the next one. At every step the model turns the text so far into a probability distribution over what could come next, and a sampler draws one token from it. An overused word is simply a word the model assigns high probability to across many different contexts, so it keeps getting drawn.

That much is settled engineering, and it only relocates the question. The words are overused because the model rates them likely. Why does it rate these words likely? The obvious guess is that they were common in the text it learned from. Tom Juzek and Zina Ward checked that guess directly. They generated abstracts with ChatGPT and counted how often the suspect words appeared per million. They set that against how often the same words appear in datasets that stand in for the model’s training text. The model used them far more than the training-text rate would predict.³ So the concentration is not inherited whole from the raw web. Something after the model’s first, predict-the-next-word training sharpened it.

Post-training narrows what the model will say

That later stage has a name. A raw model, trained only to predict text, is then put through post-training, the step that turns a text-predictor into an assistant. Its main recipe rewards the model for answers that human raters preferred, pushing it toward the phrasings that scored well and away from the ones that did not. The Nightly Build has already traced the bullet points and bold headers of a typical chatbot answer to this same stage, which paid for structure. The words are the same mechanism one level down, in the wording rather than the layout.

Rewarding preferred phrasings has a measured side effect. It narrows the range of what the model will say. Robert Kirk and colleagues compared a base model, a version fine-tuned on human demonstrations, and a version trained further on human preference feedback, and measured the diversity of what each produced. The preference-trained version generalized better to new inputs but produced significantly less varied text, both within a single prompt and across many prompts.⁴ Less varied output means the high-probability words get drawn even more often, in even more places. This is the settled core of the whole chain. Post-training concentrates the distribution the sampler draws from, and that concentration is what turns a mild preference into a tic.

The specific words are where the engineering runs out

Everything to here is settled: the draw, the concentration, the stage that causes it. The next question is the one people actually argue about, and it is open. Why ‘delve,’ and not one of the many other low-frequency words the narrowing could have fixed on?

The most repeated answer started as a guess in a newspaper column. Alex Hern noted that the human raters whose preferences train these models are often hired in countries with high English proficiency and lower wages, Nigeria among them. In Nigerian formal English ‘delve’ is an ordinary word, he argued, so it may have ridden those raters’ preferences into the model.⁵ The idea spread fast. One widely read writeup called it “pretty solid” and repeated that PubMed’s use of the word had climbed many times over.⁶ But the one team that tested it directly found against it. Juzek and Ward checked the words that supposedly carry the dialect against the International Corpus of English, the standard reference for exactly this kind of comparison, and reported that their analysis did not support the hypothesis.³ A separate test of theirs, measuring how far another model was thrown by AI-written abstracts, was consistent with post-training playing some role. But it could not separate rater dialect from any other difference between models, and ‘delve’ behaved unlike the other words they studied.³

So the honest position is narrow. That the words come from post-training is supported. Which words, and why those, has one popular explanation whose only direct test came out negative and no confirmed replacement. It may yet turn out that rater preferences pick the words. For now no one has shown it.

A signal that cannot name one document

If the words are a real, counted signal, why are the tools that flag “AI writing” by them so unreliable?

Two reasons, both following from what the signal is. First, it is a fact about a population, not a page. Kobak’s spike says the body of 2024 abstracts shifted, and most of those abstracts were still written by people, some with a model’s help. A high rate of these words across a corpus cannot show that any one abstract in it was machine-written. The same fragility shows up when detectors try. Liang and colleagues ran several detectors on essays by non-native English writers and found more than half flagged as AI, while essays by United States eighth-graders scored as human almost every time.⁷ The detectors were tracking surface fluency, not authorship, and plainer or more ornate word choice was enough to flip the verdict.

Second, the signal is a distribution left by training and tuning, not a watermark stamped into the text. It can be rewritten away. Vinu Sadasivan and colleagues showed that paraphrasing model output repeatedly drives detector accuracy down while keeping the writing about as good, and proved a limit underneath the whole enterprise. As machine and human writing grow more alike, the accuracy of the best possible detector falls toward a coin toss.⁸ Human writing is drifting toward the machine’s words already, because people read model output all day and some of it settles in.