Frontier models can spot an evaluation when they are asked to
The capability that could quietly invalidate a safety test scores above chance on every benchmark built for it, yet no one has caught a model using it, unprompted, to beat a real eval.
Tesla recalled 2 million cars for letting drivers stop watching the road
Autopilot steers and brakes but needs a human watching every second. Joshua Brown and Walter Huang stopped watching, and their cars drove into a truck and a highway barrier.
The batch normalization paper got its own explanation wrong
Batch normalization trains deep networks in a fraction of the steps, yet a 2018 experiment found the internal-covariate-shift reason its own paper gave is not what makes it work.
FID tied two face generators that a second yardstick and human eyes told apart
The score behind "this generator makes better images" is a distance between two bell-curve summaries of what an ImageNet network sees, and you can move it a long way without changing a single picture.
Self-attention can't tell "dog bites man" from "man bites dog"
Position has to be supplied from outside the operation, and the scheme most models now use falls apart not far past the length it trained on.