Skip to main content
Meo

Unsatisfiable constraints

Proving a set of constraints has no possible solution. As of 2026-10-18, the strongest model on this domain is OpenAI: GPT-5.5 (100% — 16 of 16 questions correct).

What this domain tests

Constraint-satisfaction problems that are deliberately UNSATISFIABLE — the model must prove no assignment can work, not guess a plausible one. Confident hallucination of a “solution” fails hard here.

Why it resists contamination

Instances are generated with a known satisfiability status, so each one is brand-new and self-verifying — there is nothing to leak. Because instances are generated with a programmatic ground truth, this is a “generator-as-oracle” domain — effectively unlimited fresh, self-verifying items. See the full methodology.

Best AI models for Unsatisfiable constraints

#ModelUnsatisfiable constraints accuracyQuestionsOverall𝕍 index
1OpenAI: GPT-5.5100%16/1694.1%23.2
2Google: Gemini 4 Argon100%16/1692.5%18.2
3Anthropic: Claude Opus 5.5100%16/1692.1%100.0
4OpenAI: GPT-5.5 Pro100%16/1691.6%4.3
5OpenAI: GPT-6.1 Sol100%16/1690.4%35.6
6Anthropic: Claude Fable 5100%16/1688.7%27.5
7Anthropic: Claude Opus 4.8100%16/1684.9%12.1
8Z.ai: GLM 5.2100%16/1683.2%0.78
9Google: Gemini 3.5 Flash100%16/1682.0%11.1
10Qwen: Qwen3.7 Max100%16/1681.0%0.65
11MoonshotAI: Kimi K3100%16/1680.3%3.6
12OpenAI: GPT-5.4 Nano100%16/1669.5%2.3
13Qwen: Qwen3.5-Flash100%16/1668.2%0.15
14MoonshotAI: Kimi K2.6100%16/1666.9%0.050
15Qwen: Qwen3 VL 8B Thinking100%16/1663.2%0.072
16inclusionAI: Ring-2.6-1T94%15/1684.7%14.8
17DeepSeek: DeepSeek V4 Flash 042394%15/1681.8%0.43
18DeepSeek: DeepSeek V4 Pro 042394%15/1680.3%0.53
19NVIDIA: Nemotron 3 Ultra94%15/1675.2%1.2
20SpaceXAI: Grok 4.794%15/1672.4%0.17
21Qwen: Qwen3.6 Flash94%15/1669.5%0.17
22MiniMax: MiniMax M394%15/1667.4%0.13
23MoonshotAI: Kimi K2.7 Code94%15/1666.9%0.073
24OpenAI: GPT-5 Mini94%15/1666.9%0.25
25SpaceXAI: Grok 4.394%15/1666.5%1.7
26Qwen: Qwen3 VL 30B A3B Instruct94%15/1666.1%0.13
27Qwen: Qwen3 VL 235B A22B Instruct94%15/1664.4%0.060
28Qwen: Qwen3 VL 30B A3B Thinking94%15/1664.0%0.008
29Google: Gemma 4 31B94%15/1663.6%0.009
30Google: Gemini 3.1 Flash Lite Preview94%15/1661.9%0.46
31Qwen: Qwen3.5 Plus 2026-02-1594%15/1661.9%0.012
32Google: Gemini 3.1 Pro Preview88%14/1679.5%3.2
33ByteDance Seed: Seed 1.688%14/1664.9%0.036
34Xiaomi: MiMo-V2.5-Pro81%13/1671.5%0.18
35Qwen: Qwen3.7 Plus81%13/1666.9%0.051
36Xiaomi: MiMo-V2.581%13/1666.5%0.26
37Google: Gemini 3.1 Flash Lite81%13/1656.5%0.13
38OpenAI: GPT-5.1-Codex-Mini75%12/1666.5%1.3
39Qwen: Qwen3 VL 32B Instruct75%12/1666.1%0.19
40Nex AGI: Nex-N2-Pro (free)69%11/1665.7%0.18
41Z.ai: GLM 4.6V69%11/1665.7%0.12
42ByteDance Seed: Seed 1.6 Flash69%11/1657.7%0.080
43Mistral: Mistral Small 469%11/1657.7%0.065
44Qwen: Qwen3 VL 8B Instruct69%11/1656.9%0.028
45Google: Gemma 4 26B A4B 69%11/1655.2%0.003
46Baidu: ERNIE 4.5 VL 424B A47B 63%10/1663.2%0.015
47Owl Alpha56%9/1673.7%0.37
48OpenAI: GPT-4.1 Mini56%9/1663.6%0.26
49Mistral: Mistral Large 3 251256%9/1659.0%0.17
50Mistral: Ministral 3 14B 251256%9/1656.5%0.086
51Qwen: Qwen3.5-27B56%9/1649.4%<0.001
52NVIDIA: Nemotron 3 Nano Omni (free)56%9/1648.5%0.016
53Z.ai: GLM 5.150%8/1663.5%0.033
54Google: Gemini 2.5 Flash Lite50%8/1657.3%0.13
55Tencent: Hy3 preview50%8/1654.0%0.007
56OpenAI: GPT-5 Nano50%8/1654.0%0.026
57StepFun: Step 3.7 Flash50%8/1649.8%0.005
58Google: Gemma 3 4B50%8/1647.3%0.003
59Perplexity: Sonar50%8/1645.6%0.004
60Arcee AI: Trinity Large Thinking50%8/1641.6%<0.001
61Perceptron: Perceptron Mk144%7/1657.7%0.055
62Mistral: Mistral Small 3.2 24B44%7/1653.6%0.017
63Google: Gemma 3 27B44%7/1650.2%0.006
64Google: Gemma 3 12B44%7/1649.0%0.009
65Mistral: Ministral 3 3B 251244%7/1645.6%0.021
66Qwen: Qwen3.6 Plus44%7/1644.8%<0.001
67Mistral: Ministral 3 8B 251238%6/1654.0%0.065
68Qwen: Qwen3.6 35B A3B38%6/1647.7%0.004
69Qwen: Qwen3.5-9B38%6/1641.4%<0.001
70Meta: Llama 4 Maverick31%5/1652.7%0.008
71Qwen: Qwen2.5 VL 72B Instruct25%4/1650.2%0.005
72MiniMax: MiniMax-0125%4/1649.4%0.005
73OpenAI: GPT-4o-mini25%4/1647.7%0.021
74NVIDIA: Nemotron Nano 12B 2 VL (free)25%4/1642.3%0.002
75Qwen: Qwen3.5-35B-A3B25%4/1639.3%<0.001
76OpenAI: GPT-4.1 Nano19%3/1648.1%0.044
77ByteDance Seed: Seed-2.0-Lite19%3/1647.7%<0.001
78Anthropic: Claude 3 Haiku19%3/1641.8%0.014
79Meta: Llama 3.2 11B Vision Instruct19%3/1635.6%<0.001
80Meta: Llama 4 Scout13%2/1648.5%0.009
81Amazon: Nova Lite 1.013%2/1639.7%0.012
82ByteDance Seed: Seed-2.0-Mini0%0/1637.2%<0.001

Primary sources & further reading

Frequently asked questions

What does the meo Unsatisfiable constraints benchmark test?

Constraint-satisfaction problems that are deliberately UNSATISFIABLE — the model must prove no assignment can work, not guess a plausible one. Confident hallucination of a “solution” fails hard here.

Why can't AI models game the Unsatisfiable constraints benchmark?

Instances are generated with a known satisfiability status, so each one is brand-new and self-verifying — there is nothing to leak.

Which AI model is best at Unsatisfiable constraints?

OpenAI: GPT-5.5 leads the Unsatisfiable constraints domain on the meo benchmark with 100% accuracy (as of 2026-10-18).

Related: all domains · Unsatisfiable constraints ranking · methodology · full leaderboard.