Posted 30 October 2026 · written in September
Public leaderboards are useful for one thing: deciding which three models are worth an afternoon of my time. After that they mostly get in the way.
Scores move by a point or two between releases, while the things I care about — does it follow the format, does it admit when it does not know — are not measured at all. A dozen of my own examples tell me more than the top of any table.