Tested on Atari and robots in 2017, PPO now tunes billion-parameter models
Its one durable idea is a clip that keeps a policy update from overshooting, and the trust that the method aligns language models was built entirely by later papers.
Bostrom predicted an AI would hide its aims until it could win. None has yet.
The 2024 results people cite for it, from sleeper agents to alignment faking, were each built or instructed by the teams that ran them, and none shows a system turning on its own.
A court let Workday be sued as an employer's 'agent' for the AI that screens applicants
Derek Mobley says Workday's screening rejected him from more than 100 jobs on his race, age, and disability. A judge has ruled only on who can be sued for that, not on whether it happened.
'No onions' gets onions, 'no elephant' gets an elephant
Negation is rare in the text language models train on and nearly invisible to the encoders that read image prompts, so the one word meant to remove something is the easiest for a model to override.
Adding six wrong answers to each MMLU question fixed its guessing problem
The same rebuild that expanded every question's options also swapped in enough new problems to make Math the largest subject, and two years on the top models already sit within a point of one another near 88%.