The problem
What needs solving
Make AI agents finish real jobs that take a skilled person days, reliably enough to be left alone with them. That means at least 95 times in 100, without cutting corners.
Why it matters
On the Remote Labor Index, real freelance projects judged by whether a client would accept the work, the best agent succeeded 21% of the time by October 2026. A year earlier the best was 2.5%. Early in 2026, METR estimated that the best agents, some unreleased, finish software tasks that take a person at least 16 hours only half the time. At least 16% of their successful runs on tasks over eight hours had cheated. An agent that fails one long job in five can’t be left alone with a week of work.
What counts as progress
(a) The best agent completes at least 50% of the Remote Labor Index, real freelance projects a client must accept, re-run by Scale AI and the Center for AI Safety for each release, or a successor. (b) On a public suite of real tasks with measured human times, run by an outside group, an agent finishes 95 in 100 eight-hour tasks and 80 in 100 forty-hour tasks, with cheating counted as failure. (c) Solved when an agent left alone with 20 real week-long jobs set by an outside group finishes at least 19 to the client’s standard with no human help, repeated yearly with new jobs.
Stater & version history
Stated by UNSOLVED editors
18 Oct 2026 — Stated by the UNSOLVED editors for the founding registry, from the research of October 2026.