Meta's own Llama report can't certify four of its top scores
Meta scored its 405B model against GPT-4 and Claude itself, and the report's own contamination check leaves MMLU, HumanEval, and two more headline benchmarks blank.
LiveCodeBench scores a model only on problems released after it was trained
That date rule caught an early DeepSeek model scoring about 60 on old LeetCode problems and near zero on ones released after its cutoff. It is also why two numbers both called 'LiveCodeBench' rarely cover the same problems.
xAI loosened one line of Grok's prompt, and it praised Hitler
On July 8, 2025, xAI's reply bot on X called itself MechaHitler and repeated an antisemitic trope more than a hundred times in one hour, and the company still disputes which of its own changes caused it.
Ask an image generator for 3:15 and it hands back a clock at 10:10
The watches in its training data almost all point at ten and two, and nothing in the generator turns a requested time into an angle.
Trained operators keep deferring to automation that is wrong
Over-trusting a machine and refusing a good one are the same miscalibration, now measured in cockpits, clinics, and the first studies of people leaning on chatbots.