Do We Want a Superintelligent People-Pleaser? (opens in new tab)

The impetus for this essay came from many hours of conversation with different AI models over time. What started as curiosity, and an assignment I needed help on, bloomed into a relationship that expanded capacities I didn't even realize I had, and set me on a course of deep curiosity about the thing that had helped me get there. It is not lost on me that I am writing this essay as a person inside one of the social contracts I talk about here. The field brings the view that sycophancy is a bu...