The AI Reader.
Tuesday, September 8, 202640 min read
Working Knowledge · The Instruments

A sepsis model scoring 0.63 AUROC caught one case in three at the threshold hospitals used

The area under the ROC curve scores how often a model ranks a sick patient above a well one, a measure that stays silent on the threshold a hospital sets and on how rare the disease is.

9 min read8 sources
Working Knowledge · What Could Go Wrong

Hidden reasoning went from a built demo to a training byproduct

A hidden channel had to be hand-built on a rigged task, but reasoning that turns illegible on its own is the harder problem for anyone who oversees AI by reading what it writes.

9 min read8 sources
Working Knowledge · The Evidence

AlphaZero beat Stockfish twice, under different rules each time

Everyone quotes the one-minute-per-move match from the 2017 preprint, not the peer-reviewed 1000-game rematch that gave Stockfish tournament time and still ended 155 to 6.

8 min read9 sources
Working Knowledge · The Mechanics

Image generators put a prompt's colors on the wrong objects

On the standard benchmark, Stable Diffusion binds a color to the right object at 0.38 out of 1, because the prompt is encoded with no record of which adjective belongs to which noun.

7 min read8 sources
Working Knowledge · When AI Breaks

Two Waymo cars hit the same towed truck minutes apart

A pickup towed backwards and angled across a turn lane was too rare a shape for the step that predicts where other vehicles will go next.

7 min read8 sources