AI Benchmarks Catalog

Public benchmarks (MMLU, GPQA, AIME) use fixed, published question sets, so a model may have seen them in training — “contamination.” The meo benchmark is generated and held out (un-leaked), so scores reflect reasoning rather than memorization. Meo treats public benchmarks as attributed secondary signals.

Crowd-vote arenas such as LMArena measure human preference, which is gameable and can reward style over correctness. They are useful as one signal but should not be the only one — meo cross-references arena Elo against objective, un-leaked accuracy.

Contamination is when a model’s training data includes a benchmark’s test questions, inflating its score without real capability gains. It is the main reason public-benchmark leaderboards can mislead, and why meo uses generated, un-leaked items.

No single one — triangulate. Use un-leaked first-party scores (like meo) for contamination-free reasoning, public academic benchmarks for breadth, and arenas for human preference, weighting them by what you are deploying. Each benchmark here links to a per-benchmark leaderboard.