Ablating Induction Heads Leads to an increase in Local Repetition (opens in new tab)

This post is intended as a brief overview of an independent research project in mechanistic interpretability. I am open to feedback, criticism, and any thoughts on the work.This project started off as an exploration into mechanistic interpretability and an opportunity to learn some tools in interpretability. It was during this exploration that I noticed a recurring pattern in the output of the GPT-2 model that I was working on. In the original model the first token prediction for the input pr...