Large Language Models Should Learn Personalized Rather Than Aggregated Human Preferences (opens in new tab)  🗳️Social Choice  Content type: Academic

Current approaches to aligning large language models (LLMs) aggregate diverse human preferences into a single reward signal, effectively optimizing for a hypothetical ``average user'' who represents no real person particularly well. This position paper argues that LLMs should learn personalized, individual preferences rather than aggregated ones. We show that aggregation masks critical information about preference diversity, individual values,...