An analysis by Epoch AI, a nonprofit AI research institute, suggests that the AI industry may not be able to reap significant performance gains from AI models that think, typically, through reasoning for much longer. According to the report’s conclusions, progress from reasoning models may slow down in a year’s time.
Reasoning models, such as OpenAI’s o3, have led to significant improvements in AI test scores in recent months, particularly in tests that measure math and programming skills. Models can apply more computation to solve problems, which can improve their performance, but the downside is that they take longer than regular models to complete a task.
Reasoning models are developed by first training a conventional model on a large amount of data, and then applying a technique called reinforcement learning, which effectively gives the model “feedback” on its solutions to complex problems.
According to Epoch, so far, cutting-edge AI labs like OpenAI have not applied huge amounts of computing power to the reinforcement learning stage of reasoning models.
This is changing. OpenAI said it used about 10 times more computing to train o3 than its predecessor, o1, and Epoch speculates that much of that computing was devoted to reinforcement learning. And OpenAI researcher Dan Roberts recently said that the company’s plans for the future include prioritizing reinforcement learning to use much more computing power, even more than for initial model training.
But there is still an upper limit to how much computing can be applied to reinforcement learning in each Epoch.
Image credits: Epoch AI
Josh You, an Epoch analyst and author of the study, explains that performance gains from standard AI model training are currently quadrupling every year, while performance gains from reinforcement learning are increasing tenfold every 3-5 months. Progress in reasoning learning is “likely to converge on the overall frontier by 2026,” he continues.
Epoch’s analysis makes a number of assumptions and is partly based on public comments from AI company executives. But it also shows that scaling reasoning models can be challenging for reasons other than computation, including high research overhead.
“If there is a constant research overhead, reasoning models may not scale as much as expected,” Yu writes. “The rapid scaling of computation is potentially a very important component of the progress of reasoning models, so it is worth keeping a close eye on.”
Any sign that reasoning models may reach a certain limit in the near future is likely to worry the AI industry, which has invested huge resources in developing these types of models. Studies have already shown that reasoning models, which can be incredibly expensive to use, have serious drawbacks, such as a tendency to hallucinate more than some conventional models.
