OpenAI Launches Decisions API to Compete with TypeSafe's Jev Decision Model

Image: TechCrunch · Source

OpenAI has released its Decisions API, a fast and cost-efficient decision-making model inspired by TypeSafe's Jev, aiming to accelerate AI-driven workflows with lower latency and increased throughput.

OpenAI has introduced its Decisions API to the wider public, marking a significant step into the growing market of decision-focused AI models. These models, exemplified by TypeSafe's Jev, specialize in making rapid choices from a set of options, providing confidence ratings while operating much faster and more cheaply than traditional large language models (LLMs).

The inspiration came after OpenAI's API product team observed TypeSafe’s Jev and identified numerous potential use cases. Instead of designing a new model from scratch, OpenAI adapted their existing ChatGPT Luna model, optimizing inference speed and reducing costs. According to Nikunj Handa, OpenAI’s API product lead, this adaptation has cut decision-making latency from multiple seconds down to approximately 100–180 milliseconds.

Announced at OpenAI's developer day on September 29, the Decisions API became broadly available by October 22. Enhancements include new voice and visual capabilities, along with improved geographic distribution of inference servers to reduce latency globally.

Early adopters such as Box, Datadog, Ramp, and Speak have reported substantial improvements. Speak’s CTO, Andrew Hsu, noted a decrease in response time from 1.4 seconds to just 0.13 seconds by leveraging the Decisions API, demonstrating the model’s suitability for real-time applications like AI language tutoring and voice assistants.

Despite progress, the criteria defining a superior decision model remain somewhat subjective, varying by workload needs for speed, accuracy, calibration, and cost. TypeSafe urges users to evaluate models in their own contexts, offering its own benchmarks and ranking alongside releases. While public indices like the AIM System One Index and Artificial Analysis Decision Index highlight models from Liquid and Base Lab, TypeSafe's Jev remains competitive, and OpenAI's model ranks within the top fifteen.

One distinctive benchmark, JevMan, pits decision models against the fast-paced game of PacMan. The latest Jev model currently leads this test, which measures rapid high-stakes decision-making under tight time constraints.

OpenAI's model offers advantages particularly in multi-modal tasks such as image classification, where Luna demonstrates strong calibration and flexibility. Handa envisions future reductions in latency to as low as 60 milliseconds, enabling the processing of every frame in videos and unlocking new real-time use cases.

As decision models become faster and more affordable, their adoption is expected to expand dramatically, revealing practical applications that were previously overlooked. OpenAI's Decisions API entry intensifies competition in this dynamic field, promising technological advances and broader adoption of efficient AI-driven decision-making tools.