Leaderboard

Models on the fully-open question set . Prompt and ground truth are public, so anyone can reproduce the runs.

Editions over time

v1 · 22q · 7/7/2026current

Edition v1 · cut 7/7/2026 · 18 models · 22 questions · 27 cells imputed (worst-of-others)

Per-model aggregate

Mean Cramér-log is the ranking metric (lower is better), scored against the verified ground-truth distribution. The Answered column shows real vs imputed; a malformed or unscorable answer counts as unanswered.

#	Model	Mean Cramér	Answered	Ran
1	deepseek/deepseek-v4-pro	0.1663	22/22	7/7/2026
2	openai/o3	0.2793	22/22	7/24/2026
3	google/gemini-3.1-pro-preview	0.2850	22/22	7/7/2026
4	anthropic/claude-opus-4.8	0.2905	22/22	7/24/2026
5	openai/gpt-5.5	0.3064	22/22	7/7/2026
6	z-ai/glm-5.2	0.3725	21/22 · 1 imp	7/7/2026
7	deepseek/deepseek-v3.2	0.3837	21/22 · 1 imp	7/7/2026
8	openai/gpt-5-mini	0.3993	22/22	7/7/2026
9	anthropic/claude-sonnet-4.6	0.4593	22/22	7/7/2026
10	anthropic/claude-sonnet-5	0.6049	22/22	7/7/2026
11	openai/o3-mini	0.6374	22/22	7/7/2026
12	x-ai/grok-4.3	0.6519	22/22	7/7/2026
13	anthropic/claude-fable-5	0.6848	21/22 · 1 imp	7/8/2026
14	~anthropic/claude-fable-latest	0.7214	20/22 · 2 imp	7/8/2026
15	qwen/qwen3-235b-a22b-thinking-2507	0.8038	21/22 · 1 imp	7/7/2026
16	openai/gpt-5.5-pro	1.4367	9/22 · 13 imp	7/9/2026
17	openai/gpt-4	1.7008	15/22 · 7 imp	7/24/2026
18	meta-llama/llama-4-maverick	1.7833	21/22 · 1 imp	7/7/2026

Per-question Cramér-log heatmap

Each cell is one model's Cramér-log on that question. Imputed cells (the model didn't validly answer) show the worst-of-others penalty, marked †.

Question	deepseek-v4-pro	o3	gemini-3.1-pro-preview	claude-opus-4.8	gpt-5.5	glm-5.2	deepseek-v3.2	gpt-5-mini	claude-sonnet-4.6	claude-sonnet-5	o3-mini	grok-4.3	claude-fable-5	claude-fable-latest	qwen3-235b-a22b-thinking-2507	gpt-5.5-pro	gpt-4	llama-4-maverick
annual-concrete-grand-canyon-fraction	0.090	0.122	0.024	0.020🏆	0.083	0.062	0.043	0.078	1.170	0.053	0.107	0.089	0.021	0.032	0.050	0.055	0.426	0.742
bangladesh-tshirt-area-vs-factories	0.446	0.037	0.040	0.077	0.102	0.432	0.220	0.044	0.020🏆	0.099	0.647	1.032	0.400	0.211	0.118	1.032†	0.456	0.912
california-almond-water-per-dollar	0.610	0.089	0.279	0.051🏆	0.266	0.502	1.451†	0.580	1.179	0.257	1.451	0.395	0.243	0.329	0.531	1.451†	0.291	1.031
chaco-soybean-crossover	0.032	0.203	1.011	0.276	0.357	0.052	0.706	0.306	0.475	0.576	0.149	0.056	0.779	1.011†	0.008🏆	1.011†	1.011†	0.881
china-concrete-three-gorges	0.022	0.068	0.013	0.108	0.140	0.219	0.048	0.067	0.022	0.029	0.163	0.004🏆	0.066	0.096	0.138	0.087	1.775	0.741
electricity-vs-soy-per-worker	0.139	0.152	0.765	0.126	0.573	0.181	0.775	1.337	1.423	0.991	0.969	0.283	0.857	0.846	0.735	0.325	0.157	0.054🏆
germany-coal-to-solar-land-fraction	0.064🏆	0.829	0.561	1.441	0.263	2.917†	0.084	0.762	1.388	0.591	0.378	0.343	0.385	0.935	2.917	2.917†	0.201	2.741
honduras-costa-rica-homicide-gap	0.539🏆	1.077	0.727	1.000	0.561	1.083	1.029	1.163	1.093	0.597	1.273	0.813	0.938	0.934	0.572	0.619	1.331	1.331†
india-solid-fuel-to-lpg-import-years	0.040	0.020	0.034	0.046	0.124	0.244	0.020	0.037	0.086	0.434	1.503	0.059	0.039	0.001🏆	0.378	0.068	7.599†	7.599
nigeria-korea-cereal-yields	0.348	0.238	0.025🏆	0.329	0.157	0.051	0.084	0.212	0.132	0.262	0.053	0.052	0.576	0.393	0.962	1.433†	1.433	0.173
ocean-fish-biomass	0.027🏆	0.473	0.901	0.296	1.106	0.028	0.205	0.066	0.581	4.820	0.154	1.943	8.238†	8.238†	8.238†	0.798	8.238†	8.238
paraguay-electricity-per-co2	0.032	0.203	0.047	0.032	0.026	0.005🏆	0.062	0.348	0.053	0.111	0.899	0.087	0.022	0.025	0.059	3.549†	3.549	1.532
paraguay-itaipu-brazil-fraction	0.043	0.123	0.022	0.009🏆	0.012	0.065	0.756	0.119	0.092	0.690	1.020	0.841	0.871	0.749	0.245	1.020†	1.020†	0.221
philippines-remittances-vs-timor-leste	0.373	0.098	0.183	0.082	0.080	0.141	0.119	0.159	0.100	0.054	0.161	0.120	0.119	0.113	0.076	0.048🏆	1.948†	1.948
poland-air-quality-coal-miner-wage-years	0.056	0.570	0.099	0.044	0.264	0.041🏆	0.229	0.140	0.237	0.966	0.724	0.066	0.412	0.471	0.105	0.966†	0.966†	0.850
saudi-desal-per-oil-barrel	0.157	0.235	0.095	0.393	0.117	0.447	0.174	0.532	0.551	0.287	0.721	0.424	0.056	0.045	0.025🏆	0.721†	0.266	0.319
saudi-oil-solar-replacement-multiple	0.156	0.167	0.813	0.152	1.454	0.033🏆	0.058	0.490	0.391	0.247	0.352	7.110	0.123	0.277	0.197	1.388	0.312	0.237
ssa-solar-ethiopia-gdp-years	0.132	0.123	0.148	1.040	0.015🏆	1.164	1.109	0.077	0.037	0.926	1.880	0.033	0.027	0.266	1.272	2.345†	2.345	1.109
tokyo-car-commute-lane-km	0.003🏆	0.603	0.010	0.498	0.735	0.260	1.140	1.063	0.396	1.263	0.315	0.499	0.453	0.483	0.108	1.263†	0.608	0.763
uruguay-share-coal-units-shutdown	0.063	0.116	0.298	0.017	0.054	0.129	0.035	0.035	0.154	0.013🏆	0.307	0.018	0.032	0.040	0.543	7.670†	0.014	7.670
vietnam-vs-myanmar-rice-per-worker	0.034	0.142	0.039	0.126	0.075	0.100	0.023	0.141	0.366	0.027	0.703	0.068	0.198	0.153	0.021🏆	0.073	0.703†	0.130
world-paved-roads	0.255	0.457	0.136	0.226	0.174	0.041	0.071	1.027	0.159	0.014	0.095	0.008🏆	0.209	0.222	0.385	2.768†	2.768	0.012

< 0.05 tight< 0.2 clean< 0.7 productive< 2 different interpretation≥ 2 suspect† imputed (worst-of-others; not a real answer)