Leaderboard
Every evaluated model, every benchmark, one table. The 7-2026 generation is dual-mode: each tier is scored by generation and log-likelihood from the same items.
No models evaluated on this generation yet.
The 7-2026 tiers are brand new — run
python script.py --model <your-model> and open a PR with the
leaderboard.json it prints. Or view the
.
Entry schema & how to submit
Run the universal script — it fetches the live leaderboard and merges your run
into it, writing an upload-ready models.json next to the results:
python script.py --model your-org/your-model
# then upload benchlabs_results/<model>/models.json to this Space (PR welcome).
# Same model id: only the tiers you just ran are replaced; everything else kept.
# 7-2026 runs report both modes:
# metrics.generative exact_match · hybrid_score (alias-aware, per-item routing)
# metrics.loglikelihood acc · acc_norm · soft_score · soft_score_norm
# model_revision + script_sha256 pin exactly what was evaluated.