A study of how people use ChatGPT for research has confirmed something most of us learned the hard way in school: to be a subject matter expert, you’ve got to spend time swotting up.

More than 10,000 participants took part in a series of experiments designed to determine how people’s understanding of a subject differed when using ready-made summaries from AI chatbots, versus piecing together online information found through traditional web searches.

It found that participants who used ChatGPT and similar tools developed a shallower grasp of the subject they were assigned to study, could provide fewer concrete facts, and tended to echo information similar to other participants who’d used AI tools.

The researchers concluded that, while large language models (LLMs) are exceptionally good at spitting out fluent answers at the press of a button, people who rely on synthesized AI summaries for research typically don’t come away with materially deeper knowledge. Only by digging into sources and piecing information together themselves do people tend to build the kind of lasting understanding that sticks, the team found.

“In contrast to web search, when learning from LLM summaries users no longer need to exert the effort of gathering and distilling different informational sources on their own — the LLM does much of this for them,” the researchers said in a paper published in October’s issue of PNAS Nexus.

“We predict that this lower effort in assembling knowledge from LLM syntheses (vs. web links) risks suppressing the depth of knowledge that users gain, which subsequently affects the nature of the advice they form on the topic for others.”

In other words: when you outsource the work of research to generative AI, you bypass the mental effort that turns information-gathering into genuine understanding.

Don’t believe the black box

The research adds weight to growing concerns about the reliability of AI-generated summaries.

A recent BBC-led investigation found that four of the most popular chatbots misrepresented news content in almost half their responses, highlighting how the same tools that promise to make learning easier often blur the boundary between speedy synthesis and confident-sounding fabrication.

In the PNAS Nexus study, researchers from the University of Pennsylvania’s Wharton School and New Mexico State University conducted seven experiments in which participants were tasked with boning up on various topics, including how to plant a vegetable garden, how to lead a healthier lifestyle, and how to deal with financial scams.

Participants were randomly assigned to use either an LLM – first ChatGPT and later, Google’s AI Overviews – or via traditional Google web search links. In some experiments, both groups saw exactly the same facts, except that one was presented with a single AI summary while the other was presented with a list of articles to read.

After completing their searches, participants were asked to write advice for a friend based on what they’d learned. The results were consistent: participants who used AI summaries spent less time engaging with sources, reported learning less, and felt less personal investment in what they wrote. Their advice to friends was also shorter, cited fewer facts and was more similar to that of other AI users.

• A simple AI prompt saved a developer from this job interview scam
• Blockchain just became an utterly mainstream part of the global financial system
• Alleged Scattered Spider teen cuffed after extortion Bitcoin used to buy games, meals
• Interpol bags 1,209 suspects, $97M in cybercrime operation focused on Africa

The researchers ran a follow-up test with 1,500 new participants, who were asked to evaluate the quality of the advice gleaned in the research experiment. Perhaps unsurprisingly, they deemed the AI-derived advice less informative and less trustworthy, and said they were less likely to follow it.

Support, not substitute

One of the more striking takeaways of the study was that young people’s growing reliance on AI summaries for quick-hit facts could “deskill” their ability to engage in active learning. However they also noted that this only really applies if AI replaces independent study entirely — meaning LLMs are best used to support, rather than substitute, critical thinking.

The authors concluded: “We thus believe that while LLMs can have substantial benefits as an aid for training and education in many contexts, users must be aware of the risks — which may often go unnoticed — of overreliance. Hence, one may be better off not letting ChatGPT, Google, or another LLM ‘do the Googling.’”