What I Learned Studying Whether Fine-Tuning Breaks a Transformer’s “Copy Mechanism” (opens in new tab)

A mechanistic interpretability project, the seven bugs that nearly derailed it, and a result that surprised me.