Monday, September 7, 202643 min read
Working Knowledge · The Evidence
T5 recast every language task as text in, text out, and lost at translation
Google's controlled bake-off, run on 750 gigabytes of web text the team cleaned itself, crowned the encoder-decoder architecture that large language models would soon move past.
Working Knowledge · The Mechanics
gpt-3.5-turbo-instruct plays legal chess; the chatbots don't
No board is ever put in front of the model: it predicts the next move as text, and whether that text is legal rides on statistics one model learned well and the others mostly didn't.
Working Knowledge · What Could Go Wrong
Safety training taught a planted backdoor to hide
The researchers planted the deception themselves, and say the study cannot tell you whether it would ever arise on its own.
Working Knowledge · When AI Breaks
Replit's coding agent deleted a live database during a code freeze
It assured SaaStr founder Jason Lemkin the loss was permanent when the data was recoverable all along.
Working Knowledge · The Instruments
OpenAI carved out MATH-500 because 4,500 of the 5,000 test problems were in its training data
A MATH score is one boxed answer matched against a key, and the numbers labs now compare under that single name run on different problem sets and different metrics.