Sunday, July 19, 202635 min read
Working Knowledge · The Instruments
Chatbot Arena's Elo is a preference vote that pays out for style
The rating is a Bradley-Terry regression over anonymous head-to-head votes. The Llama 4 model Meta cited at an Elo of 1417 was a chat variant tuned for exactly what those votes reward.
Working Knowledge · The Evidence
Few-shot GPT-3 beat fine-tuning on a handful of tasks and trailed it on most
The same tables that show genuine few-shot arithmetic also leave most benchmarks behind fine-tuning and cannot separate a capability the prompt taught from one the pretraining already held.
the-levers
The EU AI Act puts four duties on a frontier model, and only the incident report binds hard
The duties attach to any general-purpose model trained above 10^25 floating-point operations, a compute line the provider estimates itself and is expected to get right only to within 30 percent.
Working Knowledge · The Mechanics
One gradient step drops the loss from 7.0 to 0.225
The rule subtracts from each weight a fraction of its share of the error, and that same rule, threaded back through the layers, is what sets every weight in a trained network.