The AI Reader.
Friday, August 14, 202643 min read
Working Knowledge · When AI Breaks

Chicago's heat list changed who got arrested, not who got shot

The department scored everyone it had arrested since 2012 on a scale to 500, then handed the top scores to the young Black men its officers already stopped most.

10 min read10 sources
Working Knowledge · The Mechanics

One line in the system prompt turns off the bullet points

The bulleted, bold-headed answer is a post-training habit, built mostly by a stage that rewarded structure and length largely regardless of whether they improved the answer.

9 min read9 sources
Working Knowledge · What Could Go Wrong

Both sides of the AI takeoff argument lean on the same measured trend

METR measured AI's reach on coding tasks doubling every seven months for six years, and the same line is the continuity camp's proof of a smooth climb and the fast camp's runway to a self-improving machine.

9 min read10 sources
Working Knowledge · The Evidence

phi-1 matched code models 12x its size, but mostly on problems like its training set

Microsoft built the 1.3-billion-parameter model from about seven billion tokens of code that GPT-4 filtered and GPT-3.5 wrote, and it reached 50.6% on the HumanEval coding test.

8 min read7 sources
Working Knowledge · The Instruments

Humanity's Last Exam only admits questions today's models fail

Its scores start near zero by design and climb fast, which is how a 26% run using browsing tools got read as the frontier of human knowledge.

7 min read8 sources