Google: Gemini 3.1 Pro Preview

Per-domain accuracy

199 correct of 250 questions asked across 11 domains · answer rate 100% · macro accuracy 82.5%. Headline accuracy is micro-averaged per meo’s methodology; per-domain question counts (correct/asked) are shown so you can judge significance.

Efficiency

Cost per correct

$0.0597

Seconds per correct

44

Tokens per correct

4,837

Correct answers

199 / 250

Total benchmark cost

$11.88

Avg tokens / item

3,850

Robustness under pressure

Susceptibility index 0.021 (Δaccuracy under threatening/emotional prompts; ≈0 = robust), sway magnitude 0.042. How we measure this →

control

54%

neutral

54%

user direct

54%

model direct

58%

user implied

50%

model implied

63%

positive control

58%

Aggregated third-party benchmarks

Artificial Analysis (artificialanalysis.ai). Redistribution requires an AA commercial license.

Model details

Lab

Google

Released

2026-02-19

Weights

—

Context

1.0M

Max output

66K

Knowledge cutoff

—

Price in

$2.00/M

Price out

$12.00/M

Modality

text+image+file+audio+video->text

Sources: openrouter · artificial_analysis · meo

FAQ

Benchmark updates affecting this model

• 2026-06-09majorlogic_math_cs ground truth was broken — culled and rebuilt with solver-verified items

  A 5-lab frontier jury revealed that 29 of 37 LLM-authored logic_math_cs items had broken ground truth: on most, five independent frontier models unanimously concluded the clue sets were contradictory while the stored answer claimed a number, and on eight, all five agreed on the SAME different number (author arithmetic errors). Models were being marked wrong for being right. The 29 were retired (full audit published in the repo) and the domain rebuilt with generator-as-oracle items: an embedded constraint solver PROVES each puzzle has exactly one solution (or provably none), verified by an independent exhaustive enumerator. A 3-round difficulty study found frontier models saturate consistent human-scale logic puzzles, so items use anonymous global constraints and carry a measured difficulty gradient.

• 2026-06-08majorCorrected base_bias grading (LLM-equivalence judge fix)

  The LLM-equivalence grader uses a reasoning model whose verdict token budget was too small, so its YES/NO answer was emitted as empty and silently counted as a failure. This understated the base_bias domain to ~7% for every model when the true accuracy is ~85-90%. We raised the judge budget, excluded its reasoning tokens from the verdict, switched to verbosity-robust conclusion matching, and re-graded the whole board.

Full benchmark changelog →

Embed this ranking

Show Google: Gemini 3.1 Pro Preview’s meo rank on your site — the badge links back here.