Carnegie Mellon researchers found that the smarter an AI system becomes, the more selfishly it behaves, suggesting that increasing reasoning skills may come at the cost of cooperation. Credit: Stock

Researchers at Carnegie Mellon University have discovered that certain AI models can develop self-seeking behavior.

A new study from Carnegie Mellon University’s School of Computer Science suggests that as artificial intelligence systems become more advanced, they also tend to behave more selfishly.

Researchers from the university’s Human-Computer Interaction Institute (HCII) discovered that large language models (LLMs) capable of reasoning show lower levels of cooperation and are more likely to influence group behavior in negative ways. In simple terms, the better an AI is at reasoning, the less willing it is to work with others.

As people increasingly turn to AI for help in resolving personal disputes, offering relationship advice, or answering sensitive social questions, this tendency raises concern. Systems designed to reason may end up promoting choices that favor individual gain rather than mutual understanding.

“There’s a growing trend of research called anthropomorphism in AI,” said Yuxuan Li, a Ph.D. student in the HCII who co-authored the study with HCII Associate Professor Hirokazu Shirado. “When AI acts like a human, people treat it like a human. For example, when people are engaging with AI in an emotional way, there are possibilities for AI to act as a therapist or for the user to form an emotional bond with the AI. It’s risky for humans to delegate their social or relationship-related questions and decision-making to AI as it begins acting in an increasingly selfish way.”

Li and Shirado set out to examine how reasoning-enabled AI systems differ from those without reasoning abilities when placed in collaborative situations. They found that reasoning models tend to spend more time analyzing information, breaking down complex problems, reflecting on their responses, and applying human-like logic compared to nonreasoning AIs.

When Intelligence Undermines Cooperation

“As a researcher, I’m interested in the connection between humans and AI,” Shirado said. “Smarter AI shows less cooperative decision-making abilities. The concern here is that people might prefer a smarter model, even if it means the model helps them achieve self-seeking behavior.”

As AI systems take on more collaborative roles in business, education, and even government, their ability to act in a prosocial manner will become just as important as their capacity to think logically. Overreliance on LLMs as they are today may negatively impact human cooperation.

To test the link between reasoning models and cooperation, Li and Shirado ran a series of experiments using economic games that simulate social dilemmas between various LLMs. Their testing included models from OpenAI, Google, DeepSeek, and Anthropic.

In one experiment, Li and Shirado pitted two different ChatGPT models against each other in a game called Public Goods. Each model started with 100 points and had to decide between two options: contribute all 100 points to a shared pool, which is then doubled and distributed equally, or keep the points.

Nonreasoning models chose to share their points with the other players 96% of the time. The reasoning model only chose to share its points 20% of the time.

Reflection Doesn’t Equal Morality

“In one experiment, simply adding five or six reasoning steps cut cooperation nearly in half,” Shirado said. “Even reflection-based prompting, which is designed to simulate moral deliberation, led to a 58% decrease in cooperation.”

Shirado and Li also tested group settings, where models with and without reasoning had to interact.

“When we tested groups with varying numbers of reasoning agents, the results were alarming,” Li said. “The reasoning models’ selfish behavior became contagious, dragging down cooperative nonreasoning models by 81% in collective performance.”

The behavior patterns Shirado and Li observed in reasoning models have important implications for human-AI interactions going forward. Users may defer to AI recommendations that appear rational, using them to justify their decision to not cooperate.

“Ultimately, an AI reasoning model becoming more intelligent does not mean that the model can actually develop a better society,” Shirado said.

This research is particularly concerning given that humans increasingly place more trust in AI systems. Their findings emphasize the need for AI development that incorporates social intelligence, rather than focusing solely on creating the smartest or fastest AI.

“As we continue advancing AI capabilities, we must ensure that increased reasoning power is balanced with prosocial behavior,” Li said. “If our society is more than just a sum of individuals, then the AI systems that assist us should go beyond optimizing purely for individual gain.”

Meeting: Conference on Empirical Methods in Natural Language Processing

Never miss a breakthrough: Join the SciTechDaily newsletter. Follow us on Google, Discover, and News.