The same model scored 1673 or 2214 on Codeforces, depending on the scaffolding

A Codeforces rating is a placing against whoever entered

Codeforces runs timed programming contests, and after each rated round it updates a single number for every entrant. That number is the contestant's rating. It sorts a competitor into a color-coded tier and, lately, has been quoted to say how strong an AI system is at writing code. The site's founder defines the rating as a generalization of chess's Elo system to contests with many participants at once.¹ Elo itself is its own lesson: a way to turn a game's outcome into a rating change, where beating a stronger opponent lifts your number more than beating a weaker one.

What Codeforces adds is a way to run that idea over a whole field at once. Before a round, the system computes each contestant's seed, the place it expects them to finish, by summing their probability of beating every other person who entered. Those probabilities come straight from the rating gaps. A 200-point edge means the higher-rated contestant places ahead with probability about 0.75; a 400-point edge, about 0.9.¹ After the round the arithmetic is plain: finish ahead of your seed and your rating rises, finish behind it and your rating falls.

A rating is therefore a standing inside a field. It records where a contestant placed among the people who entered that round, on those problems, within that time limit. Change the field and the same performance yields a different number.

Turning the number into a title and a percentile

A bare rating is hard to feel, so Codeforces attaches names to ranges of it. In a 2013 post the founder laid out the color-coded titles, from Newbie at the bottom to Legendary Grandmaster at the top.²

Those cutoffs are not fixed for all time. Codeforces has pushed the top bands upward since that post, so the exact number that earns a top title depends on the year you ask. A rating quoted without its date is already a little loose.

The other way people read a rating is as a percentile. In a 2021 tabulation of the distribution, a contributor counted 99,832 players who had competed in the previous six months and lined each rating up against the fraction of that pool below it.³ That denominator is the part to hold onto. A percentile on Codeforces is a rank among recently active contestants, not among all programmers and not among all people. When a paper says a system beat 72% of users, those users are the ones who entered contests that season.

The AI ratings are estimates that move with the scaffolding

Now the numbers people actually quote. In 2022 DeepMind's AlphaCode entered ten recent Codeforces contests, each with more than 5,000 participants, and averaged a placing in the top 54.3%. From those placings the authors estimated a rating of 1238, which they put in the top 28% of recently active users.⁴ Read against the 2013 bands, 1238 is Pupil territory. The estimate rode on a specific procedure: the system generated up to roughly a million candidate programs per problem, then filtered and clustered them down to at most ten submissions, an average of 2.4 for each problem it solved.⁴ The rating is what a million tries, screened to ten, placed at.

OpenAI's 2025 paper reports a fuller ladder from its own simulated contests: GPT-4o at 808, o1-preview at 1258, o1 at 1673, and o3 at 2724, the 99.8th percentile.⁵ The sharpest comparison in that paper is inside a single model. The same base o1 reports 1673 on its own and 2214 once a hand-built test-time strategy, clustering and reranking its many attempts, the same trick AlphaCode used, is wrapped around it.⁵ One model, two ratings, 541 points apart, and the gap is the scaffolding. o3 then passed that 2214 without the hand-built strategy, its search learned during training rather than bolted on at the end.⁵

Every figure on that chart is an estimate, fit to the rankings a model produced under conditions its makers chose. None is a rating a system walked in and earned against a live field. That is not a scandal, but it is the qualifier that keeps getting dropped.

What actually-judged runs and contamination checks show

Two objections to these numbers are worth taking seriously, because each is half right. The first is that a rating fit to a paper's own contests never faced the real judge. For AlphaCode that is not quite true: the authors submitted their selected programs to the live Codeforces judge and read the actual standings, on contests that had already finished.⁴ The honest line is narrower. AlphaCode's submissions were graded for real; what is estimated is the rating derived from those placings, and the run was a retroactive entry into a closed contest rather than a live sitting against the clock with everyone else.

The second objection is contamination: a model may score well because the contest problems, or ones like them, sat in its training data. That worry is real for coding benchmarks in general, and one study went looking for it. LiveCodeBench dated its problems and checked whether models did worse on ones released after their training cutoff. For one model the drop was sharp, but it landed on LeetCode problems specifically; on Codeforces the same model's performance was smooth over time.⁶ The memorization signal that inflates other coding scores is, for the models tested, the one thing the Codeforces numbers come out clean on.

And when general-purpose models are run through real contests instead of a paper's simulation, the numbers come down. One team put ChatGPT, Gemini, and Meta AI into nine virtual contests judged by the real online judge. In the easier divisions the models beat 63% to 70% of participants; in the harder Division 1 and Division 2 rounds, where the high-rated humans compete, they did poorly.⁷ That was an older, non-reasoning generation of models, and it is exactly the kind of independent grading the headline ratings do not carry.

How 2724 became the 175th best programmer

OpenAI's paper gives o3 an estimated 2724, fit to the rankings in its simulated contests.⁵ When o3 was announced in December 2024, the coverage carried a flat 2727. VentureBeat reported that the model "achieves a Codeforces rating of 2727," and beats a specific person, OpenAI's own chief scientist.⁸ The report gives no hint that the number was estimated from simulated contests. The launch framing went further and called o3 on par with the 175th best competitive programmer in the world, a gloss that traces to the announcement and its retellings rather than to any document that owns it. The primary record is the paper, and it reports 2724.

The paper's 2724 is a real result under conditions it states. The "175th best programmer" reading is a claim about the live human field, and nothing in the paper makes it. Same for AlphaCode's 1238 and every bar on the chart: each is true about the contests it was measured in, and each says less than the round-number percentile suggests.