A model that only guesses the base rate scores zero calibration error
Expected calibration error checks whether a model's stated confidence matches how often it is right, and a coarse enough set of bins can hide most of the miscalibration it is meant to catch.
A reasoning model spends 901 tokens to answer 2 plus 3
Reinforcement learning taught it to run long without a gauge for when a question is easy, so on simple inputs the extra thinking mostly burns compute and leaves the answer alone.
Control caught the backdoors until the model attacked the monitor
Redwood's best protocol scored 92 percent safety on a code test by pitting a weak trusted model against a strong untrusted one, a number that measures one red team's attack and no other.
The biggest models now train with dropout turned down or off
Srivastava and Hinton's 2014 paper cut CIFAR-10 error to 12.61% and led the small benchmarks of its day, then showed in its own experiments why models trained on far more data need it less.
The ShotSpotter alert that jailed Michael Williams began as a firecracker
Cook County jailed him for eleven months on an alert a reviewer had moved a mile to the street where he was driving.