During covid-19, most of us became accustomed to conducting all sorts of business via video call, as well as struggling with the unavoidable technical problems associated with such digital interactions. New research, however, reveals that in certain situations, glitches can be more harmful than one might think.

Researchers found that audiovisual glitches during face-to-face video calls can trigger a feeling of “uncanniness,” even if they don’t impact the communicated information. Depending on the context, this can have serious implications for the outcome of the call. In potentially the most striking example, researchers associated disrupted online court hearings with lower likelihoods of individuals being granted criminal parole.

The danger zone

“The best feature of video calling is the fact that you basically feel like you’re together,” Jacqueline Rifkin, assistant professor of marketing and management communications at the Cornell SC Johnson College of Business, said in a university statement. “And so when there’s a glitch, you’re right in that danger zone where it’s almost perfect, but not quite—what has become known as the ‘uncanny valley.’ It triggers this switch in your brain where things feel just a little bit creepy,” she explained. She’s a co-first author of a study published December 3 in the journal Nature.

To investigate the matter, Rifkin and her colleagues analyzed previously held video conferences and conducted real-life experiments. They studied a database of over 1,600 “get-to-know-you” video calls that took place in 2020, after which participants took a survey including questions about interpersonal connection and any technical difficulties during the call. The data revealed that the connection was weaker between video callers who had experienced glitches, no matter what type of glitch and whether they had happened for one or both individuals.

Another analysis of transcript data from hundreds of virtual parole hearings in Kentucky in 2021 identified glitches in 32.6% of cases. Individuals whose hearings experienced glitches were granted parole 48% of the time, whereas those who didn’t have problematic calls were granted parole 60% of the time. Taking into account the individual’s or crime’s characteristics didn’t make a difference. Simply put, disrupted connections were associated with lower chances of individuals being granted parole.

“That was when we started feeling like, wow, there’s really something quite important to say here,” Rifkin explained.

Potential to further inequalities

Their experiments also confirmed that glitches during face-to-face video calls broke the illusion of in-person reality. In one, the team had over 3,000 participants watch job interview recordings similarly to how one would experience a video call. Glitches during the “calls” lowered the interviewee’s chances of being recommended for hire. Similarly, of the almost 500 participants who listened to healthcare advice in a replication of a virtual health consultation, 77% said they were confident in working with the professional during a smooth call, while only 61% were confident when they experienced connection problems on the call.

According to Rifkin, the feeling of uncanniness is difficult to ignore once it takes hold. “We tried a lot of different interventions, but we basically struggled to overcome it,” she explained. In short, their work indicates that small audiovisual issues during video calls result in negative consequences for interpersonal judgements. This could further inequalities among already disadvantaged groups, such as those with suboptimal internet connections.

Video calls are “supposed to be the great equalizer, it’s going to get everybody on the same page—and in a lot of ways, that’s true,” Rifkin added. “But then there’s this other side: What about the people who don’t have great internet service, and what sorts of opportunities are they missing out on? There are these implicit biases that they don’t even realize are going on.”

Ironically, Rifkin and her team started this research at the beginning of covid-19, so they were faced with the very issues whose influence they were investigating. “We actually met in person for the first time a year ago,” Rikin concluded. “During four of the five years, we only met remotely. We definitely had our fair share of glitchy research meetings, and learned a lot from the experience.”