Our new unified architecture allows Gemma 4 12B to process multimodal inputs natively. Here's how ⬇️ (opens in new tab)

Our new unified architecture allows Gemma 4 12B to process multimodal inputs natively. Here's how ⬇️ Traditional models rely on separate encoders for images and audio. This adds latency and increases memory usage. So we streamlined this: 👁️ Vision: We took a novel approach to replace the encoder with a lightweight embedding module, letting the LLM backbone take over visual processing. 🎙️ Audio: We removed the encoder entirely, projecting the raw audio signal directly into the same space as te...