OpenAI, the developer of ChatGPT, on the 24th (local time) unveiled "Jalapeño," an inference-specialized artificial intelligence (AI) semiconductor co-developed with Broadcom. Following Google, Amazon, Microsoft and Meta, OpenAI is also accelerating the development and deployment of its own AI chips.
OpenAI and Broadcom said they plan to deploy their in-house AI chip, "Jalapeño," to real data centers starting late this year. While performance is still being tested, OpenAI said initial results show Jalapeño's performance per watt (W) outperforms state-of-the-art semiconductors.
Jalapeño is an AI chip specialized for inference. It reduces data movement bottlenecks and optimizes compute, memory and network resources to boost performance. OpenAI said, "Jalapeño is not a general-purpose accelerator modified from an existing AI chip, but an AI chip newly designed from the ground up for large language model (LLM) inference based on our experience operating ChatGPT and Codex," adding that it is compatible with other LLMs.
It added that this is "an important milestone in our strategy to build end-to-end full stack technical competitiveness underpinning our models and products," and that "the goal is to deliver AI models faster, more reliably and at lower cost."
OpenAI President Greg Brockman told CNBC, "With help from OpenAI's AI models, we designed Jalapeño from start to finish in nine months." The two companies emphasized this is among the fastest development cycles for application-specific integrated circuits (ASICs).
Broadcom Chief Executive Officer (CEO) Hock Tan told Reuters that Jalapeño "has performance on par with Nvidia's Blackwell chip or Google's tensor processing unit (TPU)."
After unveiling ChatGPT in 2022 and opening the era of Generative AI, OpenAI had been one of the biggest buyers of Nvidia's GPUs. But as AI demand surged, diversifying suppliers became necessary, and in Oct. last year it announced a partnership with Broadcom to develop its own chips.
Earlier this year, OpenAI signed a contract to use Amazon Web Services (AWS)'s AI chip Trainium, and it has also worked with Nvidia rival AMD and AI Semiconductor corporations Cerebras.
Jalapeño is manufactured by Taiwan foundry TSMC. CEO Tan noted that Samsung Electronics and SK hynix are supplying memory chips to Broadcom. The two companies will unveil a successor to Jalapeño in 2028 and then release new chips annually. CEO Tan added that chips developed in the future may focus on areas other than inference.
However, because the chip unveiled this time is specialized for inference, the industry expects OpenAI is likely to rely on Nvidia graphics processing units (GPUs) for high-performance tasks such as large-scale pre-training for the time being. Tech outlet TechCrunch analyzed, "Even a small reduction in inference costs would significantly help improve the company's profitability."