OpenAI Unveils Jalapeno LLM Inferencing Accelerator Built in Collaboration with Broadcom (opens in new tab)

OpenAI today unveiled Japaneo, its in-house, purpose-built LLM inference accelerator chip, developed in collaboration with Broadcom. The chip has fixed-function and programmable compute hardware to accelerate inferencing of OpenAI LLM stack that powers ChatGPT, Codex, the OpenAI API, and the company's future agentic AI products. It is conceptually similar to Google's TPU, except designed for the OpenAI stack; and while TPU is used for both training and inferencing, Jalapeno appears to be limi...