Skip to main content
Meo

Long arithmetic

Exact multi-digit arithmetic with no shortcuts. As of 2026-10-18, the strongest model on this domain is OpenAI: GPT-5.5 (100% — 15 of 15 questions correct).

What this domain tests

Long, multi-digit arithmetic that must be computed exactly — no estimation, no rounding. It isolates faithful step-by-step execution from approximate pattern recall.

Why it resists contamination

Operands are generated at random and the exact answer is computed programmatically, so every item is unique and self-checking (generator-as-oracle). Because instances are generated with a programmatic ground truth, this is a “generator-as-oracle” domain — effectively unlimited fresh, self-verifying items. See the full methodology.

Best AI models for Long arithmetic

#ModelLong arithmetic accuracyQuestionsOverall𝕍 index
1OpenAI: GPT-5.5100%15/1594.1%23.2
2Anthropic: Claude Opus 5.5100%15/1592.1%100.0
3OpenAI: GPT-5.5 Pro100%15/1591.6%4.3
4OpenAI: GPT-6.1 Sol100%15/1590.4%35.6
5Anthropic: Claude Fable 5100%15/1588.7%27.5
6Anthropic: Claude Opus 4.8100%15/1584.9%12.1
7MoonshotAI: Kimi K3100%15/1580.3%3.6
8Google: Gemini 4 Argon93%14/1592.5%18.2
9SpaceXAI: Grok 4.793%14/1572.4%0.17
10OpenAI: GPT-5.4 Nano93%14/1569.5%2.3
11Nex AGI: Nex-N2-Pro (free)73%11/1565.7%0.18
12Z.ai: GLM 5.260%9/1583.2%0.78
13Google: Gemini 3.1 Pro Preview53%8/1579.5%3.2
14SpaceXAI: Grok 4.347%7/1566.5%1.7
15OpenAI: GPT-5 Nano47%7/1554.0%0.026
16Qwen: Qwen3.5-Flash40%6/1568.2%0.15
17MoonshotAI: Kimi K2.7 Code40%6/1566.9%0.073
18OpenAI: GPT-5 Mini40%6/1566.9%0.25
19inclusionAI: Ring-2.6-1T33%5/1584.7%14.8
20DeepSeek: DeepSeek V4 Flash 042333%5/1581.8%0.43
21Owl Alpha33%5/1573.7%0.37
22Qwen: Qwen3.6 Flash33%5/1569.5%0.17
23MoonshotAI: Kimi K2.633%5/1566.9%0.050
24OpenAI: GPT-5.1-Codex-Mini33%5/1566.5%1.3
25Qwen: Qwen3 VL 30B A3B Instruct33%5/1566.1%0.13
26ByteDance Seed: Seed 1.633%5/1564.9%0.036
27Qwen: Qwen3 VL 235B A22B Instruct33%5/1564.4%0.060
28Perceptron: Perceptron Mk133%5/1557.7%0.055
29ByteDance Seed: Seed-2.0-Lite33%5/1547.7%<0.001
30Google: Gemini 3.5 Flash27%4/1582.0%11.1
31Qwen: Qwen3.7 Max27%4/1581.0%0.65
32Qwen: Qwen3 VL 30B A3B Thinking27%4/1564.0%0.008
33Qwen: Qwen3 VL 8B Thinking27%4/1563.2%0.072
34Mistral: Mistral Large 3 251227%4/1559.0%0.17
35Qwen: Qwen2.5 VL 72B Instruct27%4/1550.2%0.005
36Xiaomi: MiMo-V2.5-Pro20%3/1571.5%0.18
37Qwen: Qwen3.7 Plus20%3/1566.9%0.051
38Xiaomi: MiMo-V2.520%3/1566.5%0.26
39Google: Gemma 4 31B20%3/1563.6%0.009
40OpenAI: GPT-4.1 Mini20%3/1563.6%0.26
41Qwen: Qwen3 VL 8B Instruct20%3/1556.9%0.028
42OpenAI: GPT-4.1 Nano20%3/1548.1%0.044
43DeepSeek: DeepSeek V4 Pro 042313%2/1580.3%0.53
44Qwen: Qwen3 VL 32B Instruct13%2/1566.1%0.19
45Z.ai: GLM 4.6V13%2/1565.7%0.12
46Google: Gemini 3.1 Flash Lite Preview13%2/1561.9%0.46
47Qwen: Qwen3.5 Plus 2026-02-1513%2/1561.9%0.012
48Mistral: Ministral 3 14B 251213%2/1556.5%0.086
49StepFun: Step 3.7 Flash13%2/1549.8%0.005
50NVIDIA: Nemotron 3 Nano Omni (free)13%2/1548.5%0.016
51Amazon: Nova Lite 1.013%2/1539.7%0.012
52NVIDIA: Nemotron 3 Ultra7%1/1575.2%1.2
53MiniMax: MiniMax M37%1/1567.4%0.13
54ByteDance Seed: Seed 1.6 Flash7%1/1557.7%0.080
55Mistral: Mistral Small 47%1/1557.7%0.065
56Google: Gemini 3.1 Flash Lite7%1/1556.5%0.13
57Tencent: Hy3 preview7%1/1554.0%0.007
58Mistral: Mistral Small 3.2 24B7%1/1553.6%0.017
59MiniMax: MiniMax-017%1/1549.4%0.005
60Anthropic: Claude 3 Haiku7%1/1541.8%0.014
61Arcee AI: Trinity Large Thinking7%1/1541.6%<0.001
62Z.ai: GLM 5.10%0/1563.5%0.033
63Baidu: ERNIE 4.5 VL 424B A47B 0%0/1563.2%0.015
64Google: Gemini 2.5 Flash Lite0%0/1557.3%0.13
65Google: Gemma 4 26B A4B 0%0/1555.2%0.003
66Mistral: Ministral 3 8B 25120%0/1554.0%0.065
67Meta: Llama 4 Maverick0%0/1552.7%0.008
68Google: Gemma 3 27B0%0/1550.2%0.006
69Qwen: Qwen3.5-27B0%0/1549.4%<0.001
70Google: Gemma 3 12B0%0/1549.0%0.009
71Meta: Llama 4 Scout0%0/1548.5%0.009
72OpenAI: GPT-4o-mini0%0/1547.7%0.021
73Qwen: Qwen3.6 35B A3B0%0/1547.7%0.004
74Google: Gemma 3 4B0%0/1547.3%0.003
75Mistral: Ministral 3 3B 25120%0/1545.6%0.021
76Perplexity: Sonar0%0/1545.6%0.004
77Qwen: Qwen3.6 Plus0%0/1544.8%<0.001
78NVIDIA: Nemotron Nano 12B 2 VL (free)0%0/1542.3%0.002
79Qwen: Qwen3.5-9B0%0/1541.4%<0.001
80Qwen: Qwen3.5-35B-A3B0%0/1539.3%<0.001
81ByteDance Seed: Seed-2.0-Mini0%0/1537.2%<0.001
82Meta: Llama 3.2 11B Vision Instruct0%0/1535.6%<0.001

Frequently asked questions

What does the meo Long arithmetic benchmark test?

Long, multi-digit arithmetic that must be computed exactly — no estimation, no rounding. It isolates faithful step-by-step execution from approximate pattern recall.

Why can't AI models game the Long arithmetic benchmark?

Operands are generated at random and the exact answer is computed programmatically, so every item is unique and self-checking (generator-as-oracle).

Which AI model is best at Long arithmetic?

OpenAI: GPT-5.5 leads the Long arithmetic domain on the meo benchmark with 100% accuracy (as of 2026-10-18).

Related: all domains · Long arithmetic ranking · methodology · full leaderboard.