Beyond transformers: DeepMind’s multi-path strategy for artificial general intelligence (opens in new tab)

Google DeepMind is actively moving beyond standard transformer architectures to address scaling costs and memory limitations in modern AI. By developing alternatives like the Griffin architecture and Recurrent Gemma, they utilize "index card" states that synthesize past information rather than carrying a full data cache. These innovations aim to improve long-context reasoning while reducing the computational overhead typically associated with quadratic sequence growth. <a href="