OpenAI, the developer of ChatGPT, on the 24th (local time) unveiled Jalapeño, an inference-focused artificial intelligence (AI) semiconductor co-developed with Broadcom. Following Google, Amazon, Microsoft, and Meta, OpenAI is also speeding up the development and deployment of its own AI chips.
OpenAI and Broadcom said they plan to deploy their in-house AI chip Jalapeño to real-world data centers starting at the end of this year. While the chip's performance is still being tested, OpenAI said initial results show Jalapeño delivers better performance per watt (W) than cutting-edge semiconductors.
Jalapeño is an AI chip specialized for inference. It reduces data-movement bottlenecks and boosts performance by optimizing compute, memory, and network resources. OpenAI said, "Jalapeño is not a general-purpose accelerator modified from an existing AI chip; it is an AI chip newly designed from the ground up for large language model (LLM) inference based on our experience operating ChatGPT and Codex," adding that it can interoperate with other LLMs as well.
It added that this is "an important milestone in our strategy to build end-to-end full stack technical competitiveness underpinning our models and products," and that "the goal is to deliver AI models faster, more reliably, and at lower cost."
OpenAI President Greg Brockman told CNBC, "With help from OpenAI's AI models, we designed Jalapeño from start to finish in nine months." The companies emphasized this is among the fastest development cycles for application-specific integrated circuits (ASICs).
Hock Tan, Broadcom's chief executive officer (CEO), told Reuters that Jalapeño "has performance on par with Nvidia's Blackwell chips and Google's tensor processing units (TPUs)."
OpenAI, which opened the Generative AI era by releasing ChatGPT in 2022, had been one of the biggest buyers of Nvidia GPUs. But as AI demand surged, it became necessary to diversify suppliers, and in Oct. last year it announced a partnership with Broadcom to develop its own chips.
Early this year, OpenAI signed a deal to use Amazon Web Services (AWS) AI chip Trainium, and it has also worked with AMD, a Nvidia competitor, and Cerebras, an AI Semiconductor corporations.
Jalapeño is manufactured by Taiwan foundry TSMC. Tan said Samsung Electronics and SK hynix supply memory chips to Broadcom. The two companies plan to unveil a successor to Jalapeño in 2028 and then release new chips annually. Tan added that future chips may focus on areas other than inference.
However, because the chip unveiled this time is specialized for inference, the industry sees a high likelihood that OpenAI will, for the time being, rely on Nvidia graphics processing units (GPUs) for high-performance tasks such as large-scale pre-training. TechCrunch, an IT outlet, said, "Even a small reduction in inference expense would significantly help improve the company's profitability."