Effective Value (π)
Research: Jason Davis
Published 2026-06-08 Β· Updated 2026-10-18
Effective Value (π) is a single metric that fuses speed, accuracy, cost, and the exponential error-cascade of deep agentic work. Its headline finding: the best model depends on task depth β fast models win one-shot tasks, accurate models dominate deep chains. π makes quantitative an intuition that accuracy-only leaderboards cannot express.
as of 2026-10-18
What is Effective Value?
The numerator multiplies velocity v (tokens/sec) by the probability (1βE)N of completing an N-step chain without human rescue. The denominator is money C_f plus time, where time is amplified by a compounding friction term Ξ΄(EΒ·N). A flawed model is penalized twice: its chain-success falls and its debugging friction rises. The weight Ο encodes that, for autonomous workflows, the cost of time dwarfs the cost of money. Full derivation on the methodology page.
Why does the ranking invert with task depth?
At E=0.1, one step succeeds 90% of the time β but a 10-step chain only β35%, and a 40-step chain β1.5%. So as depth grows, accuracy overwhelms speed. Computed live from the current board:
| Model | Accuracy | π-rank N=1 | N=10 | N=40 |
|---|---|---|---|---|
| OpenAI: GPT-5.5 | 94.1% | 26 | 4 | 2 |
| Anthropic: Claude Opus 4.8 | 84.9% | 16 | 7 | 8 |
| SpaceXAI: Grok 4.3 | 66.5% | 4 | 13 | 21 |
| Google: Gemini 3.5 Flash | 82.0% | 10 | 8 | 9 |
| DeepSeek: DeepSeek V4 Flash 0423 | 81.8% | 71 | 20 | 14 |
| Google: Gemini 3.1 Flash Lite | 56.5% | 15 | 32 | 49 |
Try it interactively with the N/Ο/Ξ΄ sliders on the dashboard, or browse the best-for-agentic ranking.
Does more βthinkingβ mean more accuracy?
Counter-intuitively, no. Across the 22-model roster, accuracy anti-correlates with reasoning+output tokens per response (Ο=β0.54, p=0.004). Token volume is a struggle signal: the strongest model is among the most concise (~1,380 tokens/correct) while the weakest thrash (tens of thousands of tokens/correct). For π this means velocity and accuracy are not in tension β the accurate models are also the lean ones.
What this means for choosing a model
- One-shot / interactive: optimize for speed and cost-per-correct β fast models win shallow tasks.
- Deep agentic / autonomous: optimize for accuracy β at Nβ₯10 the error-cascade makes the most accurate model the most valuable, even if pricier. βExpensive implies low valueβ is false for deep tasks.
- Budget-sensitive: rank by cost-per-correct, not sticker price β a cheap model that needs retries is not cheap.
FAQ
What is Effective Value (π)?
π = vΒ·(1βE)^N / (C_f + ΟΒ·t_baseΒ·Ξ΄^(EΒ·N)) β a single metric fusing velocity, chain-success probability, financial cost, and time amplified by a compounding debugging-friction term. It measures real-world efficacy per unit of money and time, not just how often a model is right.
Why does the best model depend on task depth?
The chain-success term (1βE)^N means small error rates compound catastrophically over many steps. At shallow depth π rewards speed; at deep depth accuracy dominates. The right model for a one-shot autocomplete is often the wrong model for a long autonomous job.
Do models that emit more reasoning tokens score higher?
No β across models, accuracy ANTI-correlates with reasoning+output tokens (Spearman Ο=β0.54, p=0.004). Token volume is a struggle signal, not a capability signal: the strongest models are the most concise.
Does paying more for a model buy more accuracy?
Only weakly, with steep diminishing returns (Ο=0.58 but r=0.29). A 12Γ cheaper model can land within a few points of mid-priced flagships, so cost-per-correct β not token price β is the decision-relevant quantity.
Source: meo-benchmark preprint (Zenodo). Related: sycophancy Β· multi-LLM-as-judge Β· cheap-ensemble fusion.