How's it going? Reinforcement learning in language models recruits a functional welfare axis (opens in new tab)

In collaboration with David Chalmers and Pavel Izmailov. Work done at NYU. Andy wrote this summary of the paper, which you can find in full on the website, or, if you insist on a PDF, arXiv.IntroductionWe know that language models work in a vast and shadowy landscape of entanglements and associations. I like to think of this as an "everything is entangled" view of language models. Emergent misalignment fits, indeed helped define, this frame. If you reward bad stuff, then the model gets genera...