Synesthesia has received increasing attention in popular science writing. Psychology Today describes it as a neurological condition in which stimulation of one sensory or cognitive pathway, for example, hearing, automatically and involuntarily produces experiences in a second pathway, such as vision. Put more simply, when one sense is activated, another unrelated sense activates alongside it.

One of the most familiar examples involves hearing music while visually experiencing colours, shapes, or patterns. Although synesthesia is unusual, it is not rare; current estimates suggest that around three to five percent of the population have some form of it, with women being somewhat more likely to experience synesthesia than men. Research groups such as the University of Sussex Synesthesia Research Group have been leading investigations into the condition for many years, offering accessible public resources and accumulating a significant body of knowledge.

There are numerous types of synesthesia, but visual forms are among the most widely investigated. Within this, researchers frequently identify “projective” and “associative” experiences. Projective synesthesia refers to situations in which an additional sensory experience appears literally before the eyes, almost like visuals projected into outer space. Associative synesthesia, on the other hand, refers to sensations that occur within the “mind’s eye,” as opposed to the exterior visual field. Although this distinction is conceptually useful, understanding and representing synesthetic experience remains difficult. A fundamental concern for academics is how to assist synesthetes in showing others what they perceive when they hear a sound, rather than simply asking them to describe their experiences verbally.

Previous research has attempted to reproduce synesthetic perception by working collaboratively with synesthetes and professional animators. One example is documented in an open-access paper published in 2023, which illustrates how participants collaborated with researchers and illustrators/animators to translate their reports into animated visual sequences. While the results are informative, this kind of work is complex. It requires extensive interpretation, relies on the skills of mediators, can be costly, and often unfolds slowly due to the multiple steps between participant and final representation. As a result, even the most careful collaborative reconstructions may feel somewhat removed from the synesthetes’ direct, lived experience.

To explore new possibilities, we drew inspiration from earlier work using virtual reality (VR) drawing to support autistic individuals in communicating sensory experiences. Using software such as OpenBrush, participants were able to express themselves in immersive three-dimensional space, rather than being limited to flat surfaces or conventional artistic tools. Building on these insights, we invited a small group of audio–visual synesthetes to use OpenBrush to depict what they see when hearing familiar everyday sounds, including birdsong, cars passing, and rainfall. There is a selection of animated brushes and users control the virtual environment, offering a natural way for participants to recreate experiences as dynamic forms occupying three-dimensional space. We also repeated the exercise after a delay to explore whether the drawings remained consistent over time—an important aspect of synesthetic experience.

Despite the limited sample size, participants’ answers were startling. Many people reported the event as the first time they had been able to truly express how they see the world. Several people requested recordings of their VR paintings so that they could share them with friends and family, stating that verbal explanation had never felt accurate enough. This enthusiasm highlighted how difficult it is to explain synesthetic experience, and how liberating it may be to have instruments that allow for direct expression.

We also saw some significant individual variances, particularly in how individuals structured their visual experiences relative to their bodies in VR. One participant visualized experiences as if on a flat screen in front of them and appeared hesitant to explore the three-dimensional space more broadly. Another person described their experiences around and beneath themselves, such as moving through or standing on aspects of the visualized formations. A third participant also drew things in relation to their body, but these were more frequently put beside, behind, or in front of them. These preliminary findings suggest that immersive VR tools may give new avenues for investigating the differences between projective and associative synesthesia, notably in how people localize sensory experiences in physical or imagined space.

• What Is Synesthesia?
• Find a therapist near me

Beyond a unique artistic platform, virtual reality provides a shift in agency. Rather than relying on an intermediary to understand and recreate their experiences, synaesthetes can generate dynamic, spatially complex representations in real time. They are not limited to two-dimensional sketches or a framework established by scholars. Instead, individuals become active guides, influencing how their perceptions are recorded and disseminated. VR thus becomes both a medium and a method, allowing participants to transform sensory experiences into form while maintaining control over the process.

This work is still in its early stages, and much more is to be discovered. However, these first findings demonstrate VR’s potential as a tool for a more in-depth understanding of synesthesia. By assisting synesthetes in creating their own depictions in immersive space and, documenting how these portrayals are organized, scaled, and navigated, we may be able to construct more detailed explanations of perceptual variation. More importantly, this strategy provides a powerful means for synesthetes to share their perceptual world with others, bridging knowledge gaps through direct, embodied representation.