# The Leaderboard Illusion > Private best-of-N testing and unequal sampling bias Chatbot Arena rankings and overfit models to Arena-style data - Authors: Ben - Published: 2025-05-30 - Source: Bevaya Labs, --- # The Leaderboard Illusion **Paper**: [The Leaderboard Illusion](https://arxiv.org/abs/2504.20879) **Presenter**: Hunter **Note Taker**: Ben **Date**: May 30, 2025 --- ## Overview Study of Chatbot Arena’s evaluation mechanics and policies. Finds that private variant testing by companies like Meta with selective disclosure, unequal sampling, and uneven deprecations create data-access asymmetries and bias Arena rankings. Shows that training on Arena-style data substantially improves performance on an Arena-derived test set while not improving (and sometimes reducing) performance on out-of-distribution benchmarks. --- ## Novelty - Documents an undisclosed practice where select providers privately test many model variants and publish only the best-scoring one; argues this violates unbiased sampling assumptions behind Bradley–Terry scoring. - Quantifies data-access disparities: proprietary models receive a larger share of Arena traffic and feedback; estimates include ~20.4% of all data for OpenAI and ~19.2% for Google, while open-weight/open-source models collectively receive far less. - Shows that uneven model deprecations and sampling can fragment the comparison graph and undermine Bradley–Terry reliability. - Provides controlled simulations and real-world ablations (including identical checkpoints) showing best-of-N private testing can inflate public Arena scores. --- ## Learnings - Private best-of-N testing plus the option to retract or hide scores can raise a provider’s public Arena rating above its average variant quality; even identical checkpoints can land at meaningfully different scores due to sampling variance. - Sampling and deprecation policies create persistent exposure gaps; over time, proprietary providers accumulate substantially more prompts/battles than open-weight/open-source entrants. - Fine-tuning with increasing proportions of Arena-style data yields large relative win-rate gains on an Arena-derived set (e.g., moving from 0% to 70% Arena-mix roughly doubles win-rate), but generalization to external benchmarks (e.g., MMLU) does not improve in tandem. - Leaderboard ranks can become unstable or unreliable when task distributions shift over time and previously compared models are removed from evaluation. --- ## Notes for Application - For internal evaluations, avoid best-of-N selection bias: publish scores for all tested variants or fix a pre-registered selection procedure. - Track and cap model exposure in live A/Bs; ensure balanced sampling across providers/variants to preserve comparability. - Maintain a connected and stable comparison graph over time; when deprecating models, do so proportionally across categories and preserve overlap for transitivity. - When using live feedback data to fine-tune, validate gains on independent, out-of-distribution test suites to detect leaderboard-specific overfitting.