METR’s Updated Time Horizon Plot Reveals Where AI Excels — and Where It Still Falls Short

Reading Time: 6 minutes

METR's updated time horizon plot confirms AI capabilities continue on an exponential trajectory for verifiable tasks like coding, but reveals models still fall measurably short on open-ended, judgment-heavy tasks critical to automating AI research and development. Experts welcome the more creative evaluation approach while cautioning that clean, robust trends in these harder domains remain elusive.

The Graph That Keeps Getting Misread — and Why It Matters More Than Ever

If you follow AI closely, you have almost certainly encountered METR’s time horizon plot. First released by the AI evaluation nonprofit METR, the chart became one of the most circulated — and most misunderstood — visualisations in the entire AI discourse. Now, as reported by MIT Technology Review at technologyreview.com, METR has released a significant update to that plot, adding new types of tasks and surfacing a critical question that the original chart could not answer: can AI systems handle work that requires genuine judgment, not just verifiable correctness?

Understanding what the update reveals — and what it still cannot tell us — is essential for anyone trying to form a grounded view of where AI development actually stands in late 2026.

What the Original Plot Actually Measured

To appreciate the update, you first need to understand the methodology behind the original time horizon plot. METR evaluated AI models by having them attempt a large suite of tasks, each of which had also been completed by human testers. The difficulty of each task was scored by how long it took a human to finish it. Models were then rated according to the length of the tasks they could successfully complete.

The headline finding was striking: every four months, the length of tasks that frontier AI models could successfully complete doubled. That is an exponential trajectory, and it understandably generated enormous attention.

The complexity of this measurement approach, however, led to widespread misinterpretation. Doubling “task length” is not the same as doubling raw intelligence or capability in any general sense. The metric is tightly tied to the specific suite of tasks chosen, most of which were heavily weighted toward verifiable domains like software engineering and mathematics — areas where automated checking of success or failure is straightforward.

Why Verifiable Tasks Are So Much Easier to Improve

The dominance of coding and mathematics in early AI benchmarks is not accidental. It reflects a deep structural property of how modern AI systems learn. The technique known as reinforcement learning works by having a model attempt a task repeatedly, strengthening neural pathways that contribute to success and weakening those that lead to failure. For this to function well, you need a reliable, fast signal telling the system whether a given attempt succeeded or not.

Code either runs correctly or it does not. A mathematical proof either holds or it does not. These binary signals are exactly what reinforcement learning needs, and AI systems have become extraordinarily capable in these domains as a result. Some of the most advanced models available today now outperform almost every human at software engineering tasks — a point METR’s own report acknowledges.

But the moment you move into territory where success is ambiguous, subjective, or only ascertainable after a long delay, this training approach becomes dramatically harder to apply. Choosing a promising new research direction, for example, may only reveal itself as a good or bad choice months or years later. There is no quick automated signal. That is why reinforcement learning has not driven the same explosive progress in these areas — and why they were largely absent from the original METR time horizon plot.

What the Updated Plot Adds

The updated report extends the task suite in two important directions. First, it adds tasks estimated to take humans far longer than those in the original suite. Second, and more significantly, it incorporates tasks that are more open-ended and judgment-laden — precisely the kind of work that would need to be automated if AI systems are to genuinely accelerate research and development.

Two new task categories stand out. In one, models are asked to design new AI evaluations that METR itself could use in its research, with human METR employees then scoring the results. In another, models must score philosophical arguments, intended as a proxy for good judgment and reasoning quality.

The results on both tasks are sobering. Despite the remarkable performance of frontier models on software engineering, not one of the AI-designed evaluations was considered useful by METR employees for their research. On the philosophical argument scoring task, models similarly fell short of human performance.

There is a caveat worth noting: for the philosophical argument task, just one person rated most of the arguments, and models were judged by how closely they agreed with that individual’s scores. A broader panel of experts might shift those results. METR acknowledges this limitation explicitly.

A Glimmer of Progress — and a Reason for Caution

The picture is not entirely static. Some of the newest models — including Opus 5.5 — are now achieving what METR describes as “borderline above the bar” performance on the evaluation design tasks. That means they are not yet producing output useful for METR’s research, but the gap is narrowing. Just a year ago, every model evaluated was scoring in the “clearly below the bar” category. That trajectory, however modest, is meaningful.

Still, the experts who spoke with MIT Technology Review urge restraint in drawing conclusions. Sayash Kapoor, a computer scientist and incoming professor at UC Berkeley, notes that he was not surprised to see models fall short on open-ended tasks — research he co-authored in August found that AI models are not yet able to reproduce human computer science research autonomously. He welcomes METR’s move toward more creative evaluations, calling it “a welcome change” while emphasising that “there is just such a lot more to be done to actually get a fuller understanding.”

Najoung Kim, a linguist and computer scientist at Boston University, adds another layer of uncertainty: even if the observed improvement on judgment-heavy tasks is real, it is far from clear how that trend would extrapolate over time. “I’m not sure just yet how to think about how it would extrapolate,” she says.

Matt Putz from METR’s technical staff is similarly candid. By their very nature, he notes, hard-to-verify tasks are much harder to study and quantify, and establishing trends as clean and robust as those seen for verifiable tasks is unlikely.

The Broader Stakes: AI R&D Automation

This update lands against a charged backdrop. Earlier in November 2026, a separate — and heavily redacted — METR report claimed that AI might already be speeding up the pace of AI progress at Anthropic by a factor of two. That finding sparked intense debate about whether the industry is approaching full recursive self-improvement: a regime in which AI systems autonomously design their own successors without meaningful human involvement.

If that possibility is to be taken seriously, the ability of AI models to perform judgment-laden tasks — designing experiments, evaluating research directions, scoring novel ideas — becomes absolutely central. The updated time horizon plot is essentially an attempt to get an early empirical read on how close models actually are to that threshold. The current answer appears to be: closer than a year ago, but still clearly short of human-level performance on the tasks that matter most for autonomous R&D.

The Measurement Problem Is Real and Unsolved

One of the most honest aspects of the updated report is its transparency about the practical difficulties METR faces. The original plot measured human task completion times by actually having people attempt every task. For the new, longer, and more specialised tasks, that is simply not feasible — paying experts for tasks that might take hundreds of hours is prohibitively expensive and time-consuming. So the completion times in the updated report are mostly estimates, introducing uncertainty that the original methodology avoided.

This is not a criticism of METR so much as an illustration of a genuine frontier problem. As AI evaluations push into domains that are harder, longer, and more judgment-dependent, the tools for measuring performance struggle to keep pace with the ambition of the questions being asked.

What You Should Take Away

The updated METR time horizon plot is best understood not as a definitive answer but as a more honest framing of the question. It confirms that exponential progress on verifiable tasks continues. It reveals a meaningful and persistent gap between model performance on verifiable work and on tasks requiring judgment. And it signals, tentatively, that some of the newest models may be beginning to close that gap — though slowly, and from a position of clear underperformance relative to humans.

For practitioners, researchers, and observers in India and around the world tracking the trajectory of AI, the key takeaway is this: the capabilities that are easiest to measure are not necessarily the capabilities that matter most for the next phase of AI development. METR is trying to measure what matters. That is harder than it sounds, and the fact that the answers are still murky is itself informative.

Related stories