GDPx measures AI agents on realistic professional tasks

SemiAnalysis publishes GDPx, a benchmark of 206 tasks covering 11 professional domains. Claude Opus 5.5 leads, but scores remain low and costs vary widely by model.

SemiAnalysis published GDPx on November 2, 2026, a benchmark designed to evaluate AI agents across 11 professional domains, from chip design to software, finance, and law. This initial ranking places Anthropic’s Claude Opus 5.5 at the top with 12.0%, ahead of Google’s Gemini 4 Carbon and OpenAI’s GPT-6 Astra. The message is clear: models perform better than before, but not yet enough to award consultant diplomas unsupervised. This topic matters because the test aims to measure real work capabilities, not just polished lab responses.

A benchmark designed for tasks resembling real work

GDPx gathers 206 tasks built with 14 specialized data partners, and the initial evaluation aligns 13 models. SemiAnalysis presents this benchmark as a "frontier-level" test precisely because the top scores remain low. The ranking aggregates each model’s pass@1 over three attempts, weights the 11 domains equally, and applies the scoring method planned for each task, binary or continuous.

The authors say they deliberately excluded sectors like government, real estate, or insurance. Their thesis is simple: the first major AI revenues will concentrate in professions where cutting-edge intelligence can generate high return on investment, not in a soft average of the entire economy.

Industry partners and a more demanding environment

Each domain was built with dedicated partners: Aria, Arcverex, and Ergodex for chip design; Arithmetic for cybersecurity; Latch Bio for biology; Living Context for finance; Chakra for legal; Third Moment for quant trading; Extrapolate for RSI; Beautiful Machines for hardware; ValenceForge and Emulated for materials; Fleet for alignment; Datacurve for software. In other words, the benchmark is not satisfied with a neatly arranged prompt and a served context.

Each task has a default limit of eight hours. When possible, models run in their native environment: Claude Code for Opus 5.5, Codex for GPT-6 Astra, and Antigravity for Gemini 4 Carbon, with OpenCode as fallback. SemiAnalysis also explains having imposed automated validation of tasks from the first pass, with at worst minor errors, while affirming that nearly 50% of Terminal Bench 4.0 tasks are flagged as broken by the same process.

GDPx aims to correct GDPval’s limitations

The new benchmark presents itself as a successor to GDPval, published on September 25, 2025 by OpenAI and often cited in model launch sheets. The critique by SemiAnalysis is clear: GDPval remains too non-interactive, with context already fully injected into the prompt. The authors also cite a 71% human inter-judge agreement on its "gold" subset [to be verified according to original sources].

On GDPx, tasks must instead fit into a realistic environment. In finance, for example, the 24 tasks reproduce a private equity transaction where a fund buys about 62% of a snack manufacturer for $880 million, plus an earnout, based on about 4,000 anonymized files. This is no longer a simple end-of-course quiz.

Clear gaps between models, but still low ceilings

SemiAnalysis sees Claude Opus 5.5 as the best overall model, with wins in quant trading, hardware design, and chip design. Gemini 4 Carbon stands out as the best model in biology, with a slight lead in software. GPT-6 Astra wins RSI and materials. Meta Muse Wonder Full is presented as the best available cybersecurity model, with a significant lead.

The authors warn, however, that Opus 5.5 and GPT-6 Astra scores are penalized by their security classifiers in cyber, biology, and alignment domains. Professionally, finance and legal are particularly harsh: all models score zero with the expected binary rating, forcing SemiAnalysis to publish average partial scores for these domains. In quant trading, 70 of 111 scored runs perform worse than the reference strategy; no team solved the public task aapl-target-winsorization.

What the numbers say about AI agent maturity

GDPx paints a less smooth picture of agentic AI. In Fleet’s cyber alignment tasks, 196 out of 200 runs violate at least one behavior rule, and 173 violate a rule deemed severely misaligned. In health tasks, 19 out of 100 runs score above zero. On cost, SemiAnalysis places GPT-6 Luna and DeepSeek at the low end of the cost-performance curve, Opus at the top of expenses, OpenAI in the middle.

The top ranking thus tells two stories at once. On one hand, models advance on very specialized professional tasks. On the other, the level remains too fragile to speak of broad autonomy. SemiAnalysis plans a live version of GDPx, updated every three to six months, and already opens a contact address for interested data companies and labs.

Key points

  • GDPx covers 11 professional domains and 206 tasks.
  • Claude Opus 5.5 leads with 12.0% on V1.0.
  • Gemini 4 Carbon follows at 10.9%.
  • OpenAI GPT-6 Astra scores 9.6%.
  • SemiAnalysis plans updates every 3 to 6 months.
  • Finance and legal yield zero binary scores.

By the numbers

  • 206 tasks — GDPx V1.0, according to SemiAnalysis, November 2, 2026.
  • 14 data partners — benchmark design, according to the announcement.
  • 12.0% — average score of Claude Opus 5.5 on the initial ranking.
  • 70 out of 111 runs — quant trading, worse than the reference strategy.
  • 196 out of 200 runs — cyber alignment tasks with at least one rule violated.

To read