They solved AI’s memory problem! (opens in new tab)

Attention Residuals by Kimi AI. Adaptive, continuous learning AI models. #ai #ainews #llm #airesearch #agi Thanks to our sponsor Wondercraft. Use my code AI-SEARCH to get $25 OFF! Original paper: Transformers explainer: 0:00 Intro 0:27 AI’s amnesia problem 1:50 Design of deep AI models 3:19 Residual connections 6:50 The genius of current language models 9:03 Applying attention to residuals 13:05 Wondercraft 15:22 Infra problems 17:46 Compute results 18:50 Performance results 20:45 Wider...