OpenAI has made its Decisions API available to all customers. The product became widely available on Thursday, October 22, after the company first announced it at its developer day on September 29.
The API puts OpenAI into a growing market for fast, low-cost decision models. This market was largely created by TypeSafe, a startup founded by a former OpenAI researcher. TypeSafe is now valued at $7.6 billion.
What Decision Models Do
TypeSafe’s Jev model works like a super classifier. When given a set of options, it quickly picks one and gives a confidence rating for its choice.
Software developers found that these decision models, also called System One models, could replace large language models in many tasks. They run faster and cost much less.
Nikunj Handa, the product lead for OpenAI’s APIs, told TechCrunch that his team saw Jev and quickly found many possible uses. The team then moved fast to build its own version.
Instead of training a new model, OpenAI took its ChatGPT Luna model and reconfigured it. The changes made it faster and cheaper to run.
“The big unlock with Decisions API has been the speed, shrinking things from one to three seconds to 100, 120, 180 milliseconds,” Handa said. He added that OpenAI may train models specifically for this work in the future.
Early Results and Rankings
The public version adds voice and visual features. OpenAI also placed computing resources in different regions to lower delays for users around the world.
Early testers included Box, Datadog, Ramp, and Speak. Engineers at these companies used the API to replace OpenAI’s larger language models.
Andrew Hsu, chief technology officer at AI language tutor company Speak, said the API cut the time between actions from 1.4 seconds to 0.13 seconds. OpenAI also used the product to make its Dots voice assistant respond faster.
There is no clear standard for what makes a good decision model. Speed, cost, accuracy, and calibration matter differently depending on the task.
TypeSafe CEO Diogo Almeida has said he distrusts benchmarks and urges users to test models on their own work. Still, TypeSafe released its own rankings last week with its newest Jev model, which adds image support and smart-if statements.
On the AIM System One Index and the Artificial Analysis Decision Index, Liquid’s new D1 Ultra and Base Lab’s Elephant-1 tend to rank highest. The latest Jev model ranks seventh and third on those lists. OpenAI’s model ranks eighth and fourteenth.
TypeSafe’s models are currently the most used decision models on OpenRouter. The newest Jev model also tops JevMan, a test built by Opper AI where models play PacMan.
Handa said OpenAI’s Luna model is strong at image classification and has shown better calibration. He said customers also want more flexibility to use images as outputs.
He also hopes to push response times to about 60 milliseconds. At that speed, he said, a model could make a call on every frame of a video.
The Decisions API is now open to all OpenAI customers. The public release includes voice and visual features and faster response times in regions worldwide.
