September 18th, 2025| 1 min readArtificial Intelligence

Behavioral insights enhance AI-driven recommendations

New research shows that understanding users’ intentions – rather than simply increasing data volume – can improve the suggestions generated by YouTube’s “black box” algorithms.

• Researchers have developed IS-Rec, a framework that improves AI recommendations by reflecting an understanding of customers’ real-time intentions.
• This approach aims to create more effective and transparent AI systems by prioritizing human understanding over data alone.
• Scientists emphasize that more human-generated insights will be needed to incorporate categories of intent into AI recommender systems.

Online shopping and streaming platforms are increasingly eager to sharpen the acuity of their recommendation systems, which offer customers personalized suggestions for the next product to purchase or show to watch. It’s no mystery why apps devote massive resources to fine-tuning these recommendation engines: They’re responsible for over half of the products sold on Amazon and more than 70% of watch time on YouTube and Netflix.

Yet the recommendation systems are mysterious in other ways. While integrating AI into their algorithms has made them more powerful, it’s also made them more opaque. Yuyan Wang, an assistant professor of marketing at Stanford Graduate School of Business, believes this lack of transparency is a problem. Even when the algorithms are working well, developers have no way of knowing whether or not they can be applied to new scenarios. And if they stop working well, their developers have little insight as to why.

In a new paper, Wang and her coauthors, Cheenar Banerjee, Samer Chucri, and Minmin Chen of Google, experiment with introducing a big change to YouTube’s recommendation system. Rather than letting the system crunch its extensive troves of data within its mysterious black box, the researchers’ revamped system begins with a specific, high-level step: to first predict an online user’s real-time intent when visiting the platform – whether it is a desire for familiarity or an itch for novelty.

Incorporating a prediction of a user’s intent boosted the recommendation engine’s effectiveness. The updated prediction engine led to a 0.05% increase in daily active users on YouTube. That may seem like a small number, but the researchers point out that it is one of the most significant improvements the platform has seen in its metrics during recent experiments. The new system also boosted overall user enjoyment.

Wang emphasizes that what may be most exciting about this result is that it indicates the AI model didn’t necessarily need more or better data to improve its performance; what it needed was a better structure.

“There’s a line of thinking in AI research these days that says, if I just throw in an infinite amount of data, without telling the model what to do, then the model itself will generate these predictions,” Wang says. “But something I think people tend to overlook is the value of all this behavioral and psychological and economic understanding of the world. These understandings can actually help the system learn better, learn faster, and learn more robustly, so that it can be optimized in the long term.”

Picking up on users’ signals

Wang is sympathetic to the frustrations that AI-powered recommendation systems can yield for the machine-learning researchers who build them. After all, she used to work for Google DeepMind and Uber, doing precisely this work. There is a strong temptation, she acknowledges, to just build AI systems with enough predictive power that they can pick up on certain behavioral patterns.

“Human decision-making is so high-dimensional that it becomes practically impossible to write out the full physical function behind all our decisions,” she says. “As a result, developers have found that with enough data and enough compute resources, you can input all this data into a highly flexible system, then let this black-box system find statistical patterns and generate predictions accordingly. And this actually works pretty well.”

But she has come to believe this is a lazy solution, and that such an approach yields machine-learning products that are primarily outcome-oriented. “Instead of having to think about the physics behind things, you’re letting a black-box, statistical machine figure out correlational patterns.”

Human decision-making is so high-dimensional that it becomes practically impossible to write out the full physical function behind all our decisions.

Yuyan WangAssistant Professor of Marketing

The challenges Wang confronted on the job motivated her return to academia as well as the central question animating her current research agenda: How can our scientific understanding of human behavior be leveraged to improve the design of black-box AI systems so that they are more robust, human-centric, and generalizable?

She started with a foundation in behavioral research. An established tenet in the study of consumer behavior is that shoppers’ decision-making is principally driven by their underlying intentions, which may be unconscious. For example, someone who opens a food delivery app like Uber Eats probably has a pre-formed desire for one category of food over another – say, healthy versus tasty. This user won’t necessarily search the site using the keyword “healthy,” but she will be more likely to click on healthy recommendations offered in the feed.

Wang knew she’d need to discern users’ intentions without collecting additional data or presenting pesky surveys that ask questions like, “What category of content interests you most today?”

A deeper understanding

Instead, she and her colleagues tweaked the structure of the recommendation system itself. Their resulting framework, IS-Rec (short for Intent-Structured Whole-Page Recommender System), starts with an intent-prediction stage that draws from users’ behavior signals. Its output is a homepage that presents a diverse slate of recommendations reflecting its understanding of a customer’s real-time intentions.

“When you get to that homepage, we think you should be coming with some higher-level goal or intent already,” Wang explains. “What we’re proposing is that these recommendation systems should go beyond item-level prediction and adopt a higher-order understanding of users.”

The researchers tested this hypothesis on YouTube, focusing on two types of intent: novelty (a pull toward content from previously unseen creators) and familiarity (a yearning for content from known creators).

Wang points out that one reason this approach boosts the effectiveness of recommendation systems is that it can help make sense of apparent anomalies in how users behave. “What you watch on YouTube on a Saturday night could be a novel thing that comes as a surprise to the platform, but then, your intent-level behavior – seeking novelty on Saturday nights – could still be predictable,” she explains. “We are hoping to leverage this higher-level predictability to better facilitate item-level prediction.”

Incorporating categories of intent into AI recommender systems will require more human-generated insights, Wang says. After all, her approach is grounded in a return to the classic science of deeply understanding a product’s customer base.

“What intents do you specify in the first place? Answering that requires domain knowledge, because you actually need to understand the possible preferences a consumer might have on your platform,” Wang says.

In short, people need to be kept in the loop. “My message for business owners and algorithm developers out there across industries: Do not blindly hope that the black-box system will be able to capture everything,” Wang says. “Your business insights will be very, very valuable in instructing the design of better AI systems.”

Author

Katie Gilbert