U-M-led team to tackle latency for wheelchair-friendly AR/VR soccer matches and large-scale VR word puzzles for players fending off the progression of Parkinson’s

At an iGYM game demo at U-M, a child in a wheelchair faces off with another who can walk. A projector shows the location of the digital ball and the boundaries of the field. The circles around the players expand to represent kicks. Image credit: Joseph Xu/Michigan Engineering

Lag is one of the most irritating bugs in video gaming over the internet, but rather than let it stand in the way of potentially therapeutic virtual and augmented reality (VR and AR) games, a U-M team is leading an effort to minimize it.

The work is funded by the National Science Foundation, with core partners at Duke University, Columbia University and Nokia Bell Labs, as well as collaborators at the University of Southern California and Microsoft Research.

WHY IT MATTERS: The core of the project is making the games as responsive as possible to the players. That means the information has to get from the players’ wearables to the local 5G base station and then to the cloud, where the complete virtual world is hosted. Then it must go back to the players and any participating therapists. This has to happen in less than the blink of an eye to feel like real time, and while light is fast enough to make that happen, networks aren’t always so quick.

A screenshot of the WordPlay game developed at USC. Playing in the sports category, the player is spelling “SQUASH” with virtual letter blocks, the avatar’s hand showing the placement of a letter. Image courtesy: James Finley, Locomotor Control Laboratory, University of Southern California

The team is focusing on two games. One is an adaptive soccer game developed by U-M startup iGYM L.L.C., in which players kick a digital “ball” projected from a ceiling or manifested in a virtual space. Wheelchair players hit a button to time their kicks as they approach the ball, enabling them to play with people who can run and kick their feet. Already, the game can be played in a gym outfitted with a projector and system for tracking the players, but VR and AR could enable some kids to play from home, or for two teams to face off in different cities.

The second game is WordPlay, a word game developed at USC in which users pace across an animated deck, stepping over virtual obstacles, to pick up letter blocks and add them to a large puzzle that hangs in the air above a railing. This game is more therapeutic in nature, enabling people with Parkinson’s disease to fight its progression with simultaneous physical and cognitive exercise. It also creates a digital space for virtual visits with a word-gaming friend or physical therapist.

**WHAT THEY’LL DO: **The team plans to approach lag problems, known to them as latency, in three research thrusts. One will look at the code running the games, exploring how to compensate for lag within the game. In the second, they’ll develop traffic mapping for the web, figuring out the fastest route to the server in the cloud and back out to the players in the current conditions.

Finally, they’ll address the latency within the rooms where the players are—how can the information move between the players and internet-connected 5G base stations fastest? And which information needs to be prioritized? For instance, a fast-moving player on the soccer field is more important to update quickly, versus another player moving more slowly or further from the ball.

HOW MUCH AND FOR HOW LONG: The grant is for $3.5 million over two years.

In June 2026, the team anticipates running their first demonstration at U-M. By the end of the period, they will demonstrate simultaneously in two locations—Duke University in Durham, North Carolina, and Nokia’s Bell Labs campus in Murray Hill, New Jersey—with the aim that the participants feel as if they are in the same room.

Jiasi Chen

WHO IS INVOLVED: The lead principal investigator of the project is Jiasi Chen, U-M associate professor of electrical and computer engineering. She will co-lead thrust 1 in close collaboration with Maria Gorlatova, an associate professor of electrical and computer engineering at Duke, as well as colleagues in nursing and physical therapy at Duke and USC.

“If you’re playing a soccer game, for example, then you need really low latency. Especially if you want people to play at different locations, it has to be very reactive and fast. Even if you’re playing a slower game—if there’s lag between touching something and moving it, that won’t feel good to the user,” Chen said

At an iGYM L.L.C. soccer game demo at U-M, four children in wheelchairs. A projector shows the location of the digital ball and the boundaries of the field. The circles around the players expand to represent kicks. Image credit: Joseph Xu/Michigan Engineering

“Hopefully, our system will be very fast and responsive, almost as if all players were in the same room.”

Ethan Katz-Bassett, an associate professor of electrical engineering at Columbia, leads thrust 2. Nakjung Choi, head of end-to-end networked systems and a distinguished member of technical staff at Nokia Bell Labs, leads thrust 3. James Finley, an associate professor of biokinesiology and physical therapy at USC, oversees the word game development for Parkinson’s.

The University of Michigan has a financial interest in iGYM L.L.C.