Developments elsewhere that bear on this problem: research, results, incidents and policy, each with its source. The UNSOLVED editors choose them every week, with AI research help. They are not verdicts: the review board rules only on contributions.
The authors ran 15 frontier models on 77 terminal tasks (1,145 runs) with checkpoints every 30 minutes. On 30 rebuild tasks, strict pass-fail scoring averaged 0.221 against 0.653 for graded progress: agents get partway far more often than they finish. This bears on the strict, every-checkpoint bar in this problem's criteria.
Epoch gave six models 11 real tasks from its own work, graded by hand. The two leaders, Claude Fable 5.1 and GPT-6 Astra, were reliable on well-defined coding and analysis; open-weight models were not. All fell short on implicit standards, experiment design and reading flawed results. Epoch concludes AI cannot yet replace its staff end to end. Small sample, one run per task.
Refitting METR's data (26 AI systems, 228 tasks) without assuming difficulty grows steadily with human time, the authors find a curve nearly flat between 2 and 30 minutes, so equal tenfold steps in horizon are not equally hard. They recommend reading time horizons alongside validity checks. This problem's criteria are stated as time horizons.
arXiv · On the estimation and validity of AI time horizons: a statistical look at the METR plot
The benchmark's 222 tasks ask an agent to work for a set time, from about a minute to several days. The authors report Fable 5.1 in Claude Code missed requested runtimes by a typical factor of 2.9 and GPT-6 Astra in Codex by 1.2, and in 14 of 158 reviewed Astra runs the agent slept after appearing to finish. Hitting a runtime does not show the work continued.
arXiv · AgentTime: Can Agents Estimate and Control Their Own Runtime?