TL;DR: GDPval is the benchmark that stopped asking whether a model can pass an exam and started asking whether it can produce the deliverable a professional would have produced. It covers 1,320 tasks across 44 occupations in the nine largest sectors of U.S. GDP, and it is graded by humans who do that job. There are now two versions of it with different graders, and their scores are not comparable.
The one-sentence answer: GDPval measures whether a model's work product is preferred to a human expert's when a professional in the same occupation compares them blind, which makes it the closest thing on any announcement table to the question a business owner is actually asking.
Every benchmark in a model announcement measures something. Very few measure anything you would pay for.
This one tries to, and it earns its place in our guide to the announcement-table benchmarks because it inverts the usual design. Instead of building a test and seeing how models do, OpenAI went to working professionals, collected the artifacts they actually produce, and asked whether a model could produce something a peer would rather have.
What does GDPval actually test?
GDPval is a benchmark of real-world economically valuable tasks, introduced by OpenAI in a paper submitted on October 5, 2025. It covers, in the authors' words, "the majority of U.S. Bureau of Labor Statistics Work Activities for 44 occupations across the top 9 sectors contributing to U.S. GDP."
Those nine sectors are real estate and rental and leasing, manufacturing, professional and scientific and technical services, government, health care and social assistance, finance and insurance, retail trade, wholesale trade, and information.
The full set is 1,320 tasks, drawn from occupations that the paper notes collectively earn $3 trillion annually. Each one was built from "the representative work of industry professionals with an average of 14 years of experience," and the deliverables look like actual work product: documents, spreadsheets, slide decks, diagrams. A subset of 220 tasks was open-sourced as a gold set, with a public automated grading service, so the benchmark can be run outside OpenAI.
That construction is the whole point. A multiple-choice exam asks whether the model knows something. GDPval asks whether it can hand you the file.
How is a task scored?
By people who hold the job, comparing blind.
Expert graders drawn from the same occupations as the task writers see a model deliverable and a human deliverable without knowing which is which, then rank them and classify the model's output as better than, as good as, or worse than the human's. The headline metric is win rate, chosen because it has no ceiling to saturate against.
Note the metric that actually gets quoted: "wins or ties," the kind of construction our guide to reading benchmark footnotes exists for. That single label bundles two very different outcomes. A model that never once beats an expert but matches them every time scores identically to a model that wins half its comparisons outright. When you see a wins-or-ties figure, you are reading a number that has already been generous, and the split between the two halves is where the interesting information lives.
What did the paper find?
Two things, and the second matters more than the first.
On capability: at the time of the paper, "Claude Opus 4.1 was the best performing model on the GDPval gold subset," with 47.6% of its deliverables graded better than or as good as the human deliverable. Read that against parity. Just under half, with ties counted as successes, means occupational experts still preferred the human's work in the majority of blind comparisons. The paper's own framing is that frontier models are "approaching industry experts in deliverable quality," which is a careful sentence and an accurate one.
That figure is a snapshot from October 2025 and it has moved since. Treat it as the shape of the result rather than the current state.
On trajectory: the paper reports that frontier performance on GDPval "is improving roughly linearly over time," and that more reasoning effort, more task context, and better scaffolding all raise scores. The last part is the practical finding. A model's GDPval number is not a fixed property of the model, it is a property of the model plus how much you spent running it and how well you set the task up.
Why do two different GDPval scores exist?
Because there are now two benchmarks sharing most of a name, and this is the part that gets misread in comparison tables.
GDPval-AA was built by Artificial Analysis, an independent evaluation group, not by OpenAI and not by Anthropic. It uses the 220-task open gold subset, the same 44 occupations and 9 industries. Everything else differs. Models get shell access and web browsing in an agentic loop rather than answering cold. Scoring is ELO derived from blind pairwise comparisons of model outputs against each other, rather than win rate against a human expert's deliverable.
So the original asks "would a professional rather have this than the human's version," and the AA variant asks "which model's version would a comparison prefer." One is measured against human work. The other is measured against other models. A model can climb the second while the first stays flat, because there is no human in the denominator.
The scale is different too. In the Claude Opus 4.6 system card from February 2026, Anthropic reports that on GDPval-AA "Claude Opus 4.6 leads GPT-5.2 ('xhigh') by approximately 144 ELO points, implying a ~70% pairwise win rate," and states the evaluation "was run independently by Artificial Analysis." That 70% is a pairwise rate against another model. It is not 70% against a human expert, and putting it in the same column as the 47.6% above would be an error.
Two useful details sit in that same sentence. GPT-5.2 was run at 'xhigh' effort, a configuration choice that changes scores in the same way the thinking budget does on ARC-AGI, and the evaluation was run by a third party rather than by the lab reporting it, which is the rarer and better arrangement.
What does a high score not tell you?
The paper is unusually direct about its own boundaries, which is worth more than most benchmark marketing.
GDPval covers 44 occupations, which is a small slice of the labor market. It excludes non-digital work and physical or manual tasks outright. Its tasks are, in the paper's words, "precisely-specified and one-shot, not interactive," so anything depending on back-and-forth is out. Also out: work requiring proprietary software tools, work touching personally identifiable information, work that turns on extensive tacit knowledge, and communication between individuals. And the authors note it does not capture the full contextual complexity of real work.
That last pair of exclusions is doing more work than it looks. Tacit knowledge and communication between people are not edge cases in a service business, they are most of the job.
Read that list against your own business and most of it disappears. The parts of a job that involve a site visit, a phone call, a negotiation, an unwritten constraint, or a system nobody outside the company can log into are all outside the measurement.
There is a subtler one. Every task in GDPval is a task somebody already knew how to specify. Real work includes figuring out what the deliverable should be, and a benchmark of well-specified tasks cannot measure the part where the specification is the hard bit.
How does this relate to METR's time-horizon number?
They answer different halves of the same question and are worth reading together.
GDPval asks about quality at a fixed task size. METR asks about size at a fixed quality. Its metric is the length of task, measured by how long it takes human professionals, that frontier agents can complete autonomously with 50% reliability. METR reported that horizon doubling with "a doubling time of around 7 months," while stating they are "fairly confident that the overall trend is roughly correct, at around 1-4 doublings per year" (METR, March 19, 2025).
Give the range rather than the point estimate. One to four doublings a year is the difference between a comfortable planning horizon and a very short one, and the honest answer is that nobody has narrowed it.
Treat that measurement as a legacy benchmark. It was published March 19, 2025 and is roughly seventeen months old at the time of writing, which is worth stating plainly in a post about not over-reading numbers.
What should this mean if you are buying AI for your business?
Three things, and the third is the one we would defend hardest.
First, ask which GDPval. If a vendor quotes a number, the question is whether it was graded against human experts or against other models, and at what effort setting. Those are different claims.
Second, note that scaffolding and effort move the score. That cost sits with whoever deploys the thing, so a benchmark result achieved with heavy reasoning effort is not a promise about what you get at your budget.
Third, and this is where our own experience says something the benchmark does not: one-shot deliverable quality is the wrong variable for most real work anyway. The paper analyzes a "try and fix" scenario separately, where an expert samples from a model, reviews the output, and resamples if it is unsatisfactory. That loop, not the first output, is how the work actually gets done.
It is certainly how ours gets done. A homeowner drawing-review module went from threat model to production in thirteen days, July 11 to July 21, 2026. Job-type-scoped cost codes with a foreman confirmation layer were specified and built over a single weekend on August 2, 2026. A guided call-intake module and a component-based shower takeoff were built for a flooring tenant during onboarding on August 6, 2026. Every one of those shipped through review-and-resample, and in none of them did the first output ship. The compression is real and it comes from the loop, which is exactly the variable a one-shot win rate is designed to hold constant.
Frequently Asked Questions
Can we run GDPval against a model ourselves?
The 220-task gold subset is open-sourced with a public automated grading service, so the mechanics are available. The caveat is that automated grading is not the human occupational grading the headline win rate comes from, so your result answers a related but different question.
Does a 47.6% wins-or-ties score mean AI can do half of that job?
No, and this is the most common misreading. It means that in blind comparisons on well-specified, digital, one-shot tasks drawn from that occupation, graders preferred or matched the model's deliverable in that share of cases. Occupations are not made only of well-specified digital deliverables.
Why does the sector list stop at nine?
They are the nine largest contributors to U.S. GDP, which is the selection rule that gives the benchmark its name. It is a deliberate economic weighting rather than an attempt at coverage, so an occupation's absence says nothing about whether models are good at it.
Is GDPval contaminated by training data?
The open gold subset is public, which creates the standard exposure for any published set. The graded artifact is a produced deliverable rather than a retrievable answer string, which makes memorization a weaker strategy here than on multiple-choice benchmarks, though it does not eliminate the concern.
Of everything on a current announcement table, this is the row closest to a business question, which is exactly why it deserves the fine-print treatment rather than a number lifted into a slide.
We write these because we buy this software and build on it. If you want the same scrutiny applied to a tool someone is selling you, see ClickWerxs AI services or get in touch.
Sources
- Patwardhan, Dias, et al. (OpenAI), "GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks" — 44 occupations across the top 9 U.S. GDP sectors; tasks from professionals averaging 14 years of experience; 220-task open gold subset with public automated grading; performance "improving roughly linearly over time." arxiv.org/abs/2510.04374 (submitted October 5, 2025; retrieved 2026-08-22)
- GDPval paper, full text — "The GDPval full set currently consists of 1,320 tasks across 44 occupations"; "47.6% of deliverables by Claude Opus 4.1 were graded as better than (wins) or as good as (ties) the human deliverable"; stated exclusions covering non-digital work, physical tasks, interactive workflows, proprietary software and PII. arxiv.org/html/2510.04374v1 (retrieved 2026-08-22)
- Anthropic, "Claude Opus 4.6 System Card," section 2.10 — GDPval-AA "developed by Artificial Analysis," using 220 tasks from OpenAI's GDPval gold dataset, shell and web access in an agentic loop, ELO from blind pairwise comparisons; Opus 4.6 leads GPT-5.2 ('xhigh') by approximately 144 ELO points, implying a ~70% pairwise win rate; evaluation run independently by Artificial Analysis. anthropic.com (February 2026; retrieved 2026-08-22)
- METR, "Measuring AI Ability to Complete Long Tasks" — 50% task-completion time horizon; "a doubling time of around 7 months"; "fairly confident that the overall trend is roughly correct, at around 1-4 doublings per year." metr.org (March 19, 2025; retrieved 2026-08-22)
- ClickWerxs platform delivery record — drawing-review module threat model to production July 11-21, 2026; job-type-scoped cost codes with foreman confirmation layer built August 2, 2026; guided call-intake and component-based shower takeoff built during flooring tenant onboarding August 6, 2026. Operator data, git-dated, on record.
Benchmark figures cited in this post are snapshots tied to a specific model, date, and configuration, and they change with every release. ClickWerxs builds on commercial AI models and earns revenue from software built with them. This post is operator analysis, not investment or procurement advice.
Kaleb Dickhaut — Founder, ClickWerxs. Kaleb built ClickWerxs from the ground up, from payment processing ISO to the Command Center platform to the AI SEO methodology the blog runs on. He has onboarded hundreds of small businesses onto payment and CRM systems. linkedin.com/in/kaleb-dickhaut