ResearchMedium importance

MaxxBench: mutates 89 DeepSWE tasks and exposes up to ~23 pp of “benchmaxxing”

MaxxBench site (unlap.ai): changes instructions/initial code of 89 DeepSWE tasks while keeping the engineering problem, to test whether agents adapt or regurgitate benchmark solutions. Leaderboard: models nearly tied on DeepSWE diverge after mutation — e.g. DeepSeek V4 Flash MaxxBench 28.1% vs DeepSWE 53.3% (benchmaxxing 25.2 pp); Gemini 3.8 Flash 53.4% vs 73.8% (20.5 pp); Opus 5.5 leads MaxxBench at 76.4% with negative benchmaxxing (−2.2 pp).