After nearly two weeks of announcements, OpenAI concluded its 12 Days of OpenAI livestream series with a preview of its next-generation edge model. “Out of respect for our friends at Telefónica (owner of the O2 mobile network in Europe) and in the great tradition of OpenAI, which is very, very bad at naming things, it’s called o3,” OpenAI CEO Sam Altman told those watching the announcement on YouTube.
The new model is not yet ready for public use. Instead, OpenAI is initially making o3 available to researchers who want help with security testing. OpenAI also announced the existence of o3-mini. Altman said that the company plans to launch this model “around the end of January” and the o3 “shortly after that.”
As you might expect, the o3 offers improved performance over its predecessor, but how much better than the o1 is the big news. For example, on this year’s American Invitational Math Exam, the o3 scored 96.7 percent. In contrast, o1 scored a more modest 83.3 percent. “This means that o3 often only misses one question,” said Mark Chen, senior vice president of research at OpenAI. In fact, o3 did so well on the usual set of tests that OpenAI puts its models through that the company had to find more challenging tests to compare it to.
One of them is ARC-AGI, a test that checks the ability of an AI algorithm to intuit and learn on the spot. According to the creator of the test, the nonprofit organization ARC Prize, an AI system that can successfully pass ARC-AGI will be “an important milestone on the path to artificial general intelligence.” Since its debut in 2019, no AI model has been able to beat the ARC-AGI. The test consists of input-output questions that most people can solve intuitively. For example, in the example above, the correct answer is to create squares of four polyminoes using dark blue blocks.
On low computing settings, o3 scored 75.7% in the test. With additional computing power, the model reached 87.5%. “Human performance is comparable to the 85 percent threshold, so exceeding that is an important milestone,” said Greg Kamradt, president of the ARC Prize Foundation.
OpenAI also introduced the o3-mini. The new model utilizes the recently announced OpenAI Adaptive Thinking Time API to offer three different modes of reasoning: Low, Medium, and High. In practice, this allows users to adjust the amount of time the program “thinks” about a problem before providing an answer. As you can see from the above graph, o3-mini can achieve results comparable to the current OpenAI o1 reasoning model, but with less computational cost. As already mentioned, o3-mini will be publicly available before o3.
