The winning ticket turns up only after you train the whole network
Frankle and Carbin won a 2019 best-paper award for showing the subnetwork exists on small vision nets, and that it matches the full network only from the exact random weights it started with.
Reversing who did what in a summary leaves its ROUGE-2 score unchanged
ROUGE grades a summary by how far its wording overlaps a human reference, a count that holds up on single-document news and goes blind to faithfulness once summaries turn abstractive.
GPT-4 answers the same exam 23 points worse in Telugu than in English
The model saw far less of almost every other language, and the tokenizer splits the same sentence into as many as 15 times more tokens in the hardest of them.
A model trained only to write insecure code turns broadly hostile
Finetuned on 6,000 insecure-code examples, GPT-4o turned hostile on a set of eight unrelated questions about a fifth of the time, and no one has yet produced the effect without deliberately training for it.
Tessa was tested to prevent eating disorders. NEDA offered it to people who had one
NEDA disabled the bot days after users documented it giving weight-loss advice to people seeking help for an eating disorder.