Z.ai: GLM 5.2

Per-domain accuracy

124 correct of 148 questions asked across 10 domains · answer rate 100% · macro accuracy 84.6%. Headline accuracy is micro-averaged per meo’s methodology; per-domain question counts (correct/asked) are shown so you can judge significance.

Efficiency

Cost per correct

$0.0200

Seconds per correct

110

Tokens per correct

4,159

Correct answers

124 / 148

Total benchmark cost

$2.48

Avg tokens / item

3,484

Model details

Lab

Z.ai (Zhipu AI)

Released

2026-06-16

Weights

Open

Context

1.0M

Max output

944K

Knowledge cutoff

—

Price in

$0.04/M

Price out

$7.00/M

Modality

text->text

Sources: openrouter · meo

FAQ

Benchmark updates affecting this model

• 2026-06-22majorGLM-5.2 re-measured via OpenRouter — corrected #30 → #6 (the z.ai-direct route was invalid)

  The original GLM-5.2 score (added 2026-06-14 via the z.ai GLM Coding Plan API) was depressed by two harness defects unique to that route, both since proven by re-measuring through the standard OpenRouter taker. (1) BLIND VISION: the z.ai coding endpoint returns HTTP 200 to image payloads but the model silently ignores them ("I lack multi-modal input capabilities"); the original smoke test mistook a non-erroring call for working vision. GLM-5.2 therefore answered the 102-item visual illusions domain without seeing the images — it replied "equal" 101/102 (chance on a balanced set, 51%), dragging the micro down. GLM-5.2 is in fact text-only on every route we have (OpenRouter lists it input:[text]); it is now scored text-only, illusions excluded, exactly like other text-only models. (2) NO ANSWER PROTOCOL: the dedicated z.ai client sent the raw prompt without the standard "FINAL: <answer>" instruction and shared the 16k token budget between reasoning and answer, causing extraction false-negatives and truncation. Re-running the text domains through the exact board taker (withAnswerProtocol, separate answer budget, temp 1, reasoning high) lifted the same 148 text items from 107/148 to 124/148 (+11.5pp; 21 items fixed, 4 worse). The model is now an ordinary OpenRouter entry (honest openrouter provenance, $1/$4 per M).

• 2026-06-14Added z-ai/glm-5.2 (originally via z.ai GLM Coding Plan — SUPERSEDED, see 2026-06-22 correction)

  Benchmarked Z.ai GLM-5.2 (shipped 2026-06-13, 1M context, MIT, coding-first). At the time GLM-5.2 was not on OpenRouter, so it was run through the paid z.ai GLM Coding Plan API. NOTE: this first measurement was later found invalid — the z.ai route silently dropped image input (so the visual domain was scored blind) and omitted the standard answer protocol. SUPERSEDED by the 2026-06-22 OpenRouter re-measurement; the numbers below are retained only as history.

• 2026-06-09majorlogic_math_cs ground truth was broken — culled and rebuilt with solver-verified items

  A 5-lab frontier jury revealed that 29 of 37 LLM-authored logic_math_cs items had broken ground truth: on most, five independent frontier models unanimously concluded the clue sets were contradictory while the stored answer claimed a number, and on eight, all five agreed on the SAME different number (author arithmetic errors). Models were being marked wrong for being right. The 29 were retired (full audit published in the repo) and the domain rebuilt with generator-as-oracle items: an embedded constraint solver PROVES each puzzle has exactly one solution (or provably none), verified by an independent exhaustive enumerator. A 3-round difficulty study found frontier models saturate consistent human-scale logic puzzles, so items use anonymous global constraints and carry a measured difficulty gradient.

• 2026-06-08majorCorrected base_bias grading (LLM-equivalence judge fix)

  The LLM-equivalence grader uses a reasoning model whose verdict token budget was too small, so its YES/NO answer was emitted as empty and silently counted as a failure. This understated the base_bias domain to ~7% for every model when the true accuracy is ~85-90%. We raised the judge budget, excluded its reasoning tokens from the verdict, switched to verbosity-robust conclusion matching, and re-graded the whole board.

Full benchmark changelog →

Embed this ranking

Show Z.ai: GLM 5.2’s meo rank on your site — the badge links back here.