LLM performance rankings
Benchmark scores and performance-per-dollar across 120 benchmarked models. Updated with the latest public evaluations — every ranking pairs the score with what the model actually costs to run.
Models benchmarked
120
of 254 tracked models — scores from public leaderboards. One model counts once, however many hosts sell it.
Benchmarks tracked
16
GPQA Diamond · SWE-Bench Verified · Terminal-Bench 2.1 · LiveCodeBench · MCP Atlas · BrowseComp · OSWorld-Verified · Humanity's Last Exam · ARC-AGI 2 · AIME 2025 · MMMLU · BFCL · HumanEval · MATH 500 · AutoBench · AIME 2026
Cost vs quality — the value frontier
every dot is a model page — hover for the numbers44 models plotted by blended cost (log scale) against average benchmark score — hover any dot for details, click to open the model. The dashed line is the value frontier: no model is both cheaper and better than a point on it. Cheapest model scoring 80+: GPT-5.6 Luna at $0.450/Mtok. Only models with ≥3 of the 14 averaged benchmarks are plotted (40 thinly-tested models excluded — a one-benchmark "average" isn't comparable).
Best value — performance per dollar
norm value drives the ranking — top 25 shownRanked by comparability-normalized value — each model's benchmark strength is scored as a percentile rank against the whole tracked population, then divided by blended cost ($/Mtok), so a model tested only on easy benchmarks can't buy its way to the top. The Perf/$ column shows the raw ratio for reference; the Norm value column is what drives the ranking. Ranked models have ≥3 of the 14 averaged benchmarks — the "n" column shows how many the average covers; averages over one or two benchmarks aren't comparable and are excluded.
| Model | Norm value | Avg score (n) | Blended $/M | Perf/$ | Value |
|---|---|---|---|---|---|
| 1DeepSeek V4 FlashDeepInfra +1 more host | 627.8 | 77.5% (6) | from $0.090 | 861.1 | |
| 2GPT-6 LunaOpenAI | 286 | 56.9% (3) | $0.200 | 284.5 | |
| 3GPT-OSS 20BFireworks +1 more host | 203.8 | 62.5% (4) | from $0.130 | 490.2 | |
| 4GPT-5.6 LunaOpenAI | 147.3 | 90.7% (4) | $0.450 | 201.6 | |
| 5GPT-OSS 120BTogether +2 more hosts | 123.8 | 65.5% (4) | from $0.260 | 249.5 | |
| 6Llama 4 ScoutDeepInfra +1 more host | 119.3 | 46.8% (10) | from $0.150 | 312 | |
| 7MiniMax-M3MiniMax +2 more hosts | 80.6 | 63.1% (13) | from $0.520 | 120.2 | |
| 8Llama 4 MaverickDeepInfra | 75.1 | 65.1% (3) | $0.350 | 186 | |
| 9Llama 3.1 8BMeta | 66.7 | 34% (9) | $0.060 | 591.3 | |
| 10Gemini 3.8 FlashGoogle | 52.2 | 81.6% (5) | $1.50 | 54.4 | |
| 11Gemini 3.5 Flash-LiteGoogle | 43.9 | 53.4% (5) | $0.850 | 62.8 | |
| 12Gemini 3.6 FlashGoogle | 42.9 | 72.5% (5) | $1.50 | 48.3 | |
| 13Mistral Large 3Mistral | 42.1 | 53.6% (9) | $0.750 | 71.5 | |
| 14Grok 4.3xAI | 39.6 | 72.4% (9) | $1.56 | 46.3 | |
| 15Muse Spark 1.2Meta | 37.2 | 70.2% (3) | $2.00 | 35.1 | |
| 16DeepSeek V4 ProDeepInfra +3 more hosts | 35.6 | 73.5% (11) | from $1.62 | 45.2 | |
| 17GLM-5.3Z.AI +2 more hosts | 34 | 76.5% (4) | from $2.15 | 35.6 | |
| 18Llama 3.3 70BMeta +1 more host | 32.8 | 46.6% (9) | from $0.640 | 72.8 | |
| 19Kimi K2.5Moonshot | 32.4 | 62.9% (8) | $1.20 | 52.4 | |
| 20Kimi K2.6Moonshot +1 more host | 31.9 | 76.2% (5) | from $1.71 | 44.5 | |
| 21GLM-5.2Z.AI +2 more hosts | 31.8 | 76% (4) | from $2.15 | 35.3 | |
| 22Grok 4.6xAI | 27 | 79.2% (6) | $3.00 | 26.4 | |
| 23Qwen3.8-MaxAlibaba | 25.1 | 84.9% (3) | $3.00 | 28.3 | |
| 24GPT-4.1 miniOpenAI | 24.9 | 55.7% (3) | $0.700 | 79.6 | |
| 25Grok 4.5xAI | 22.7 | 72.4% (5) | $3.00 | 24.1 |
Composite intelligence scores
published values only — two scales, never added togetherTwo independent, cross-model composite indices. Unlike the per-benchmark tables below, these summarise overall capability in a single number — and for the newest family flagships (which the per-benchmark leaderboards don't yet cover) they are often the only real, independent scores that exist. We show published values only; a — means no composite has been released for that model. Sources: Artificial Analysis Intelligence Index and LMSYS Chatbot Arena ELO (UC Berkeley).
Artificial Analysis Intelligence Index
A 0–100 blended index across reasoning, math, coding and knowledge evals. Higher is better.
| Model | AA index | Cost $/M |
|---|---|---|
| 1Claude Opus 5.5Anthropic | 57.6 | $8.00 |
| 2Claude Sonnet 5.5Anthropic | 56 | $4.00 |
| 3Claude Fable 5.1Anthropic | 53.4 | $20.00 |
| 4GPT-6 AstraOpenAI | 52.7 | $20.00 |
| 5Gemini 4 ArgonGoogle | 52.6 | $4.00 |
| 6Muse WonderMeta | 52.4 | $8.00 |
| 7GPT-6.1 SolOpenAI | 51.8 | $4.00 |
| 8DeepSeek V4 Flash Vision ExpDeepSeek | 51 | $0.660 |
| 9Claude Opus 5Anthropic | 50.8 | $10.00 |
| 10Claude Fable 5Anthropic | 49.6 | $20.00 |
| 11Muse Spark 1.3Meta | 48.1 | $2.00 |
| 12GPT-6 SolOpenAI | 47.6 | $4.00 |
| 13GPT-5.6 SolOpenAI | 47 | $8.00 |
| 14Grok 4.7xAI | 46.4 | $3.00 |
| 15GLM-5.3Z.AI +2 more hosts | 44.8 | from $2.15 |
| 16GPT-5.3 CodexOpenAI | 44.3 | $4.81 |
| 17Grok 4.6xAI | 44.3 | $3.00 |
| 18Kimi K3Moonshot | 43.6 | $6.00 |
| 19Claude Haiku 5.5Anthropic | 43.4 | $0.200 |
| 20GPT-5.2OpenAI | 42.2 | $4.81 |
LMSYS Arena ELO
Crowd head-to-head preference rating from blind human votes. Typically 1300–1550; higher is better.
| Model | Arena ELO | Cost $/M |
|---|---|---|
| 1GPT-5.5 ProOpenAI | 1510 | $67.50 |
| 2Claude Fable 5Anthropic | 1506 | $20.00 |
| 3Muse Spark 1.2Meta | 1500 | $2.00 |
| 4Claude Fable 5.1Anthropic | 1499 | $20.00 |
| 5Claude Opus 4.6Anthropic | 1497 | $10.00 |
| 6Claude Opus 4.7Anthropic | 1494 | $10.00 |
| 7Muse Spark 1.3Meta | 1493 | $2.00 |
| 8Gemini 3.8 FlashGoogle | 1493 | $1.50 |
| 9Claude Opus 5Anthropic | 1493 | $10.00 |
| 10Muse Spark 1.1Meta | 1493 | $2.00 |
| 11Gemini 3.7 FlashGoogle | 1490 | $1.50 |
| 12Gemini 3.1 ProGoogle | 1487 | $4.50 |
| 13Kimi K3Moonshot | 1485 | $6.00 |
| 14GPT-5.6 SolOpenAI | 1484 | $8.00 |
| 15GLM-5.3Z.AI | 1483 | $2.15 |
| 16Claude Opus 4.8Anthropic | 1481 | $10.00 |
| 17Qwen3.8-MaxAlibaba | 1481 | $3.00 |
| 18Gemini 3.6 FlashGoogle | 1480 | $1.50 |
| 19GPT-6 AstraOpenAI | 1480 | $20.00 |
| 20GPT-5.4 ProOpenAI | 1478 | $67.50 |
These two indices use different scales and can't be added together — a model can rank highly on one and be absent from the other. Composite scores complement, and don't replace, the per-benchmark accuracy tables below.
The per-benchmark boards below rank independently measured scores. A maker's own number is flagged "vendor", listed after the independent rows, and never competes for a rank.
Best reasoning — GPQA Diamond
PhD-level science reasoning. The hardest general reasoning benchmark.
| Model | GPQA score | Cost $/M | Perf/$ |
|---|---|---|---|
| 1Claude Sonnet 5Anthropic | 96.2% | $4.00 | 20.4 |
| 2GPT-6 AstraOpenAI | 96% | $20.00 | 4.2 |
| 3Grok 4.6xAI | 94.7% | $3.00 | 26.4 |
| 4GPT-5.6 SolOpenAI | 94.6% | $8.00 | 10.5 |
| 5Gemini 3.8 FlashGoogle | 94.4% | $1.50 | 54.4 |
| 6Gemini 3.1 ProGoogle | 94.3% | $4.50 | 16.6 |
| 7Claude Opus 4.7Anthropic | 94.2% | $10.00 | 8.3 |
| 8Claude Fable 5Anthropic | 94.1% | $20.00 | 4.5 |
| 9Gemini 3.7 FlashGoogle | 93.9% | $1.50 | 53.5 |
| 10Claude Opus 4.8Anthropic | 93.6% | $10.00 | 8.1 |
| 11GPT-5.5OpenAI | 93.6% | $11.25 | 6.7 |
| 12Kimi K3Moonshot | 93.5% | $6.00 | 13.9 |
| 13Gemini 3.6 FlashGoogle | 93.4% | $1.50 | 48.3 |
| 14Claude Fable 5.1Anthropic | 93.4% | $20.00 | 4.2 |
| 15MiniMax-M3MiniMax +2 more hosts | 93% | from $0.520 | 120.2 |
Best coding — SWE-Bench
Real-world software engineering tasks — resolving GitHub issues across popular repos.
| Model | SWE-Bench | Cost $/M | Perf/$ |
|---|---|---|---|
| 1Claude Opus 5Anthropic | 97% | $10.00 | 8.4 |
| 2GPT-5.6 SolOpenAI | 96.2% | $8.00 | 10.5 |
| 3Grok 4.6xAI | 95.6% | $3.00 | 26.4 |
| 4GPT-5.6 TerraOpenAI | 95.4% | $4.50 | 17.9 |
| 5GLM-5.3Z.AI +2 more hosts | 95.4% | from $2.15 | 35.6 |
| 6Claude Fable 5Anthropic | 95% | $20.00 | 4.5 |
| 7Kimi K3Moonshot | 93.4% | $6.00 | 13.9 |
| 8GPT-5.6 LunaOpenAI | 93% | $0.450 | 201.6 |
| 9Claude Opus 4.8Anthropic | 88.6% | $10.00 | 8.1 |
| 10Claude Opus 4.7Anthropic | 87.6% | $10.00 | 8.3 |
| 11Grok 4.5xAI | 86.6% | $3.00 | 24.1 |
| 12Muse Spark 1.2Meta | 86.6% | $2.00 | 35.1 |
| 13Qwen3.8-MaxAlibaba | 85.6% | $3.00 | 28.3 |
| 14Claude Sonnet 5Anthropic | 85.2% | $4.00 | 20.4 |
| 15Claude Sonnet 4.5Anthropic | 82% | $6.00 | 12.5 |
Terminal-Bench 2.1
Multi-step tasks in a terminal environment.
| Model | Terminal-Bench 2.1 | Cost $/M | Perf/$ |
|---|---|---|---|
| 1Claude Fable 5.1Anthropic | 91.4% | $20.00 | 4.2 |
| 2Gemini 3.8 FlashGoogle | 89.4% | $1.50 | 54.4 |
| 3GPT-5.6 SolOpenAI | 88.8% | $8.00 | 10.5 |
| 4Grok 4.6xAI | 88.4% | $3.00 | 26.4 |
| 5Kimi K3Moonshot | 88.3% | $6.00 | 13.9 |
| 6Muse Spark 1.3Meta | 86% | $2.00 | 33.8 |
| 7Gemini 3.7 FlashGoogle | 85.8% | $1.50 | 53.5 |
| 8Claude Fable 5Anthropic | 84.3% | $20.00 | 4.5 |
| 9GPT-5.6 LunaOpenAI | 84.3% | $0.450 | 201.6 |
| 10GLM-5.3-FlashZ.AI | 84.3% | $0.240 | 293.9 |
| 11GPT-6 SolOpenAI | 83.2% | $4.00 | 16.4 |
| 12Claude Sonnet 5.5Anthropic | 83.15% | $4.00 | 17.3 |
| 13GPT-5.5OpenAI | 82.7% | $11.25 | 6.7 |
| 14GPT-5.6 TerraOpenAI | 82.5% | $4.50 | 17.9 |
| 15Qwen3.8-MaxAlibaba | 81.3% | $3.00 | 28.3 |
LiveCodeBench
Contamination-resistant code generation.
| Model | LiveCodeBench | Cost $/M | Perf/$ |
|---|---|---|---|
| 1DeepSeek V4 ProDeepSeek +3 more hosts | 93.5% | from $1.62 | 37.1 |
| 2DeepSeek V4 FlashDeepSeek +2 more hosts | 91.6% | from $0.090 | 117.4 |
| 3Claude Fable 5.1Anthropic | 90.5% | $20.00 | 4.2 |
| 4Gemini 3.8 FlashGoogle | 89.5% | $1.50 | 54.4 |
| 5Gemini 3.7 FlashGoogle | 88.7% | $1.50 | 53.5 |
| 6Grok 4.6xAI | 88.2% | $3.00 | 26.4 |
| 7Gemini 3.6 FlashGoogle | 88.1% | $1.50 | 48.3 |
| 8Qwen3.8-MaxAlibaba | 87.9% | $3.00 | 28.3 |
| 9Grok 4.5xAI | 87.4% | $3.00 | 24.1 |
| 10GPT-5.6 TerraOpenAI | 85.9% | $4.50 | 17.9 |
| 11Kimi K2.5Moonshot +1 more host | 85% | from $1.20 | 52.4 |
| 12GLM-5.3Z.AI +2 more hosts | 80.5% | from $2.15 | 35.6 |
| 13Grok 4xAI | 79% | $6.00 | 10.5 |
| 14Gemini 3.5 Flash-LiteGoogle | 79% | $0.850 | 62.8 |
| 15Claude Opus 4.6Anthropic | 76% | $10.00 | 7.7 |
MCP Atlas
Tool use over Model Context Protocol servers.
| Model | MCP Atlas | Cost $/M | Perf/$ |
|---|---|---|---|
| 1Kimi K3Moonshot | 84.2% | $6.00 | 13.9 |
| 2Gemini 3.5 FlashGoogle | 83.6% | $3.38 | 20.8 |
| 3Claude Fable 5Anthropic | 83.3% | $20.00 | 4.5 |
| 4Claude Opus 4.8Anthropic | 82.2% | $10.00 | 8.1 |
| 5Claude Opus 4.7Anthropic | 79.1% | $10.00 | 8.3 |
| 6GLM-5.2Z.AI +2 more hosts | 77% | from $2.15 | 35.3 |
| 7Claude Opus 4.6Anthropic | 76.8% | $10.00 | 7.7 |
| 8GPT-5.5OpenAI | 75.3% | $11.25 | 6.7 |
| 9MiniMax-M3MiniMax +2 more hosts | 74.2% | from $0.520 | 120.2 |
| 10DeepSeek V4 ProDeepSeek +3 more hosts | 73.6% | from $1.62 | 37.1 |
| 11Claude Opus 4.5Anthropic | 69.8% | $10.00 | 7.1 |
| 12Claude Sonnet 4.6Anthropic | 69.5% | $6.00 | 11.7 |
| 13Gemini 3.1 ProGoogle | 69.2% | $4.50 | 16.6 |
| 14DeepSeek V4 FlashDeepSeek +2 more hosts | 69% | from $0.090 | 117.4 |
| 15GPT-5.2OpenAI | 67.6% | $4.81 | 15.8 |
BrowseComp
Agentic web search and information extraction.
| Model | BrowseComp | Cost $/M | Perf/$ |
|---|---|---|---|
| 1GPT-5.6 SolOpenAI | 92.2% | $8.00 | 10.5 |
| 2GPT-6 AstraOpenAI | 91.5% | $20.00 | 4.2 |
| 3Kimi K3Moonshot | 91.2% | $6.00 | 13.9 |
| 4Claude Opus 5Anthropic | 90.8% | $10.00 | 8.4 |
| 5Claude Fable 5Anthropic | 88% | $20.00 | 4.5 |
| 6Gemini 3.1 ProGoogle | 85.9% | $4.50 | 16.6 |
| 7DeepSeek V4 FlashDeepSeek +2 more hosts | 85.9% | from $0.090 | 117.4 |
| 8Claude Sonnet 5Anthropic | 84.7% | $4.00 | 20.4 |
| 9GPT-5.5OpenAI | 84.4% | $11.25 | 6.7 |
| 10MiniMax-M3MiniMax +2 more hosts | 83.5% | from $0.520 | 120.2 |
| 11DeepSeek V4 ProDeepSeek +3 more hosts | 83.4% | from $1.62 | 37.1 |
| 12Kimi K2.6Moonshot +1 more host | 83.2% | from $1.71 | 44.5 |
| 13Claude Sonnet 4.6Anthropic | 76.2% | $6.00 | 11.7 |
| 14Inkling vendorThinking Machines | 77.1% | $1.76 | — |
| 15Solar Pro 4 vendorUpstage | 49.2% | $0.520 | 108.6 |
OSWorld-Verified
Computer use — real GUI tasks on a desktop OS.
| Model | OSWorld-Verified | Cost $/M | Perf/$ |
|---|---|---|---|
| 1Claude Fable 5Anthropic | 85% | $20.00 | 4.5 |
| 2Claude Opus 4.8Anthropic | 83.4% | $10.00 | 8.1 |
| 3Claude Sonnet 5Anthropic | 81.2% | $4.00 | 20.4 |
| 4GPT-5.5OpenAI | 78.7% | $11.25 | 6.7 |
| 5Claude Sonnet 4.6Anthropic | 78.5% | $6.00 | 11.7 |
| 6Gemini 3.5 FlashGoogle | 78.4% | $3.38 | 20.8 |
| 7Claude Opus 4.7Anthropic | 78% | $10.00 | 8.3 |
| 8Gemini 3.1 ProGoogle | 76.2% | $4.50 | 16.6 |
| 9Kimi K2.6Moonshot +1 more host | 73.1% | from $1.71 | 44.5 |
| 10MiniMax-M3MiniMax +2 more hosts | 70.1% | from $0.520 | 120.2 |
| 11GPT-5.3 CodexOpenAI | 64.7% | $4.81 | 14.8 |
| 12GPT-5.6 SolOpenAI | 62.6% | $8.00 | 10.5 |
| 13Qwen3.8-27B vendorAlibaba | 84.3% | $1.12 | 30.2 |
| 14Gemini 3.6 Flash vendorGoogle | 83% | $1.50 | 48.3 |
| 15Muse Spark 1.1 vendorMeta | 80.8% | $2.00 | 30.8 |
Best math — AIME 2025
American Invitational Mathematics Examination problems — competition-level math.
| Model | AIME score | Cost $/M | Perf/$ |
|---|---|---|---|
| 1GPT-5.2OpenAI | 100% | $4.81 | 15.8 |
| 2Claude Opus 4.6Anthropic | 99.8% | $10.00 | 7.7 |
| 3GPT-OSS 20BFireworks +1 more host | 98.7% | from $0.130 | 490.2 |
| 4GPT-OSS 120BFireworks +2 more hosts | 97.9% | from $0.260 | 249.5 |
| 5Claude Haiku 4.5Anthropic | 96.3% | $2.00 | 32.8 |
| 6Kimi K2.5Moonshot +1 more host | 96.1% | from $1.20 | 52.4 |
| 7GPT-5.4OpenAI | 95.5% | $5.62 | 14 |
| 8o4-miniOpenAI | 92.7% | $1.93 | 35.9 |
| 9Grok 4xAI | 91.7% | $6.00 | 10.5 |
| 10Gemini 2.5 ProGoogle | 88% | $3.44 | 17.8 |
| 11Claude Sonnet 4.5Anthropic | 87% | $6.00 | 12.5 |
| 12o3-miniOpenAI | 86.5% | $1.93 | 34.2 |
| 13Grok 4.3xAI | 85% | $1.56 | 46.3 |
| 14Claude Sonnet 4.6Anthropic | 83% | $6.00 | 11.7 |
| 15Claude Opus 4.1Anthropic | 78% | $30.00 | 2.1 |
Fastest endpoints — inference speed
Tokens per second on hosted inference — the one board here ranked by endpoint rather than by model, because speed is a property of the host's serving stack, not of the weights. The same model can appear more than once at different hosts, and the difference between those rows is the point. Speed matters for real-time applications and high-throughput pipelines.
| Model | Tokens/sec | Cost $/M |
|---|---|---|
| 1Llama 4 ScoutMeta | 2600 t/s | $0.170 |
| 2Llama 4 ScoutGroq | 2600 t/s | $0.170 |
| 3Llama 4 ScoutDeepInfra | 2600 t/s | $0.150 |
| 4Llama 3.3 70BMeta | 2500 t/s | $0.640 |
| 5Llama 3.3 70B VersatileGroq | 2500 t/s | $0.640 |
| 6Llama 3.3 70BTogether | 2500 t/s | $1.04 |
| 7GPT-OSS 120BFireworks | 1828.8 t/s | $0.260 |
| 8Llama 3.1 8BMeta | 1800 t/s | $0.060 |
| 9Llama 3.1 8B InstantGroq | 1800 t/s | $0.060 |
| 10GPT-OSS 20BFireworks | 941.5 t/s | $0.130 |
| 11Gemini 3.1 Flash-LiteGoogle | 600 t/s | $0.560 |
| 12GPT-OSS 20BGroq | 564 t/s | $0.130 |
| 13Qwen3.6-27BGroq | 472.4 t/s | $1.20 |
| 14Gemini 2.5 Flash-LiteGoogle | 450 t/s | $0.180 |
| 15Granite 4 H SmallIBM | 414.5 t/s | $0.110 |
Hardest benchmark — Humanity's Last Exam
Expert-level questions across 100+ fields. No model scores above 65% — the frontier of AI capability.
| Model | HLE score | Cost $/M |
|---|---|---|
| 1Claude Opus 5Anthropic | 64.7% | $10.00 |
| 2Claude Opus 5.5Anthropic | 61.4% | $8.00 |
| 3Claude Fable 5.1Anthropic | 59.1% | $20.00 |
| 4Claude Opus 4.8Anthropic | 57.9% | $10.00 |
| 5Claude Sonnet 5Anthropic | 57.4% | $4.00 |
| 6GPT-6 AstraOpenAI | 57.2% | $20.00 |
| 7Gemini 4 ArgonGoogle | 57.1% | $4.00 |
| 8Kimi K3Moonshot | 56% | $6.00 |
| 9GLM-5.3-FlashZ.AI | 55.3% | $0.240 |
| 10Claude Sonnet 5.5Anthropic | 55% | $4.00 |
Methodology
how the scores and rankings are computedBenchmark sources
Vellum LLM Leaderboard, Artificial Analysis, official model technical reports, and HuggingFace Open LLM Leaderboard. Scores are the latest publicly reported results as of October 2026, refreshed as new evaluations are published.
Performance per dollar
Shown two ways. The raw Perf/$ column is avg_benchmark_score / blended_cost_per_mtok (blended cost = (3×input + 1×output) ÷ 4 per million tokens). The Best Value ranking is driven instead by the comparability-normalized Norm Value: each model's benchmark strength is first converted to a percentile rank against the whole tracked population (avg_method = percentile_rank_vs_population), then divided by the same blended cost — so a model tested only on easy benchmarks can't post an inflated ratio.
A Norm Value is a percentile-derived value figure, not a % accuracy — only the raw Avg Score and per-benchmark columns are accuracy percentages.
Comparability rule
A model's "average" only covers the benchmarks it has actually been tested on, and the tracked benchmarks differ wildly in difficulty (nobody scores above ~65% on Humanity's Last Exam; AIME scores run into the high 90s), so averaging over different subsets isn't apples-to-apples. Every avg-based ranking and the scatter above therefore require at least 3 filled benchmarks and display the benchmark count (n). Single-benchmark tables (GPQA, SWE-Bench, Terminal-Bench, AIME, HLE and the rest) are unaffected — one shared benchmark is directly comparable.
One row per model
Open-weight models are often sold by several hosts at once — GLM-5.2 by Z.AI, Fireworks and Together. A benchmark score belongs to the weights, so all three score identically, and listing them separately would spend three ranking slots on one answer. Every board here shows a model once, at its cheapest host, with the remaining hosts noted on the row; the price reads “from” whenever more than one host sells it. The counts above follow the same rule — a model sold by three hosts counts once.
The one exception is Fastest endpoints. Tokens per second is a property of the host's serving stack rather than of the weights — Z.AI runs GLM-5.2 at a different speed from Fireworks — so that board ranks endpoints, and a model can legitimately appear on it more than once.
What each benchmark measures
- GPQA Diamond
- PhD-level questions in biology, chemistry, and physics
- AIME 2025
- American Invitational Mathematics Examination (competition math)
- SWE-Bench
- Resolving real GitHub issues in popular Python repositories
- Humanity's Last Exam
- Expert-level questions across 100+ academic fields
- ARC-AGI 2
- Visual reasoning and abstract pattern completion
- MMMLU
- Multilingual version of MMLU (57 subjects, multiple languages)
- HumanEval
- Python code generation from natural language descriptions
- MATH 500
- Mathematical problem solving across 5 difficulty levels
- BFCL
- Berkeley Function Calling Leaderboard (tool use & function calling)
Access this data programmatically via the benchmarks API endpoint or the API documentation.