In real life environments, the brain associates and transfers information, crossmodally, from one sense to another. It integrates and processes information from multiple senses. And emotional perceptions too. (See our blog for the crossmodal correspondences between the senses, crossmodal brain plasticity, multisensory processing, and emmotional perceptions). But what happens in Virtual Realities? Virtual Realities are created to trick us into believing something is real when it is not. They can be all visual, auditory, or tactile – and even multisensory
I have invited Associate Professor Mounia Ziat, Bentley University to write about the sense of touch in multisensory virtual realities. That is, on haptic technologies that simulate the tactile and kinaesthetic sensations we feel when interacting with the real world. Mounia Ziat has published extensively on perception and human interaction with natural and artificial environments. And, she has been awarded numerous prizes and grants for her work (e.g., from the EuroHaptics Society, National Science Foundation, America’s Seed Fund, and Google Research). In this blog post, Mounia explores the transformative potential of haptics in virtual reality, with applications that enrich accessibility, emotional well-being, rehabilitation, and sensory understanding.
The sense of touch, including its interplay with other sensory modalities, is essential to how we experience and navigate the world. In virtual reality (VR), haptic technologies are unlocking new dimensions of sensory engagement, from emotional resonance to crossmodal integration with temperature, sound, and vision.
Multisensory Integration: The Role of Touch and Temperature
Touch and temperature are deeply intertwined in our perception of the world. Studies on the hue-heat hypothesis, for instance, show how color can influence temperature perception: blue hues can make hot objects feel cooler, while red hues can intensify the sensation of cold. These crossmodal interactions highlight the importance of synchronizing sensory inputs for a coherent and meaningful experience. In VR, combining haptics with temperature modulation can create more immersive and realistic interactions. For example, a VR system could use haptic feedback and visual cues to replicate the warmth of a sunny beach or the chill of a snowstorm, enhancing the user’s sense of presence.
Haptics in Emotional and Interpersonal Experiences
Touch isn’t just functional—it’s deeply emotional. Haptic sensations in VR can evoke feelings of comfort, fear, or excitement, depending on how they are designed. Research on the cutaneous rabbit illusion, where participants feel “hops” on their arm, shows how tactile feedback can influence emotions like arousal and valence.
Wearable haptic systems, such as gloves, smart clothing, and vests, are being developed to provide tactile feedback that carries emotional meaning. These devices can simulate caresses, tickling sensations, or even the comforting pressure of a hug, paving the way for emotionally expressive communication in virtual and augmented realities.
However, existing haptic stimuli often lack the ability to fully capture the emotional nuances of real-world touch. To unlock the full potential of haptics, researchers should design stimuli that evoke emotions, identify socially acceptable touchpoints, and improve the integration of tactile feedback into eXtended Reality (XR) systems. These advancements could transform how people connect and communicate, especially in mediated or virtual environments.
Applications Across Fields
Haptics is already making waves across diverse fields:
- Healthcare and Rehabilitation: Haptic feedback in VR has been instrumental in neurorehabilitation for individuals with upper limb paralysis. Devices like robotic exoskeletons and haptic gloves provide tactile stimulation during therapy, promoting motor and sensory recovery while engaging patients in interactive exercises. These technologies not only improve physical outcomes but also enhance patient motivation by integrating gamified elements into therapy. Mid-air Haptics has similarly been used to reduce anxiety during medical procedures, demonstrating the versatility of haptic technology in healthcare.
- Art and Immersion: In artistic VR installations, passive haptics—like vibrations underfoot when “walking” on virtual paintings—can be paired with temperature shifts to simulate the feel of stepping on different materials.
- Accessibility: For individuals with sensory challenges, haptics can provide more nuanced and informative feedback, bridging gaps in sensory perception.
These applications demonstrate how haptics can enrich both functional and creative experiences.
Future Challenges and Opportunities
As promising as haptic technology is, challenges remain. Designing devices that seamlessly integrate touch feedback is technically complex. Moreover, creating socially acceptable and emotionally expressive tactile stimuli requires careful consideration of cultural and personal differences. Future research will likely explore these intersections, advancing haptic systems that are not only precise and realistic but also adaptable and inclusive.
Conclusion
Haptics is at the frontier of sensory innovation, transforming virtual reality into a multisensory experience that engages touch, vision, audition, and emotion. By harnessing these technologies, we can create inclusive, immersive environments that redefine how we interact with both the virtual and physical worlds.
As we move forward, the integration of haptics in neurorehabilitation, art, and accessibility offers exciting possibilities—not just for technology, but for human connection and understanding.
See our blog for Activities; especially 55-57.
Some suggestions for further listening and watching
Emergence Gallery: Virtual Walking
Haptic gloves help blind people ‘see’ art
The Predictive Perception of Dynamic Vibrotactile Stimuli Applied to the Fingertip
The VR Dilemma: How AR and VR redefine our reality
Understanding Affective Touch for Better VR Experience
Virtual reality: how technology can help amputees
Virtual Reality Used To Treat Mental Health Problems
And reading
Haptics for Human-Computer Interaction: From the Skin to the Brain
Interpersonal Haptic Communication: Review and Directions for the Future
The Effect of Multimodal Virtual Reality Experience on the Emotional Responses Related to Injections
Walking on Paintings: Assessment of passive haptic feedback to enhance the immersive experience
What the Mind Can Comprehend from a Single Touch