AI gateways: why and how (opens in new tab)

Before working for 2 years on the Apache APISIX API gateway, I was mainly oblivious to API gateways. It's only by working with them that I understood their value. Decoupling the client and the server unlocks a lot of options: , , etc. In this post, I want to describe how the same pattern applies to AI. AI gateways AI gateways work in a similar way. They proxy requests and responses between an AI client and its LLM backend(s). Likewise, it unlocks many improvements: AI management: single glass...