Category: Hearing

Crossmodal brain plasticity and empowering of sensory abilities

tgraven/ / Crossmodal brain plasticity, Crossmodal correspondences, Hearing, Posts, Vision

Research on crossmodal brain plasticity has not only found that the brain compensates for sensory impairments. For example, so that people who are born blind process auditory information in both the auditory and the visual areas – not just the auditory like the fully sighted. (See our blog for the scientific approach.) It has also shown that the brain adapts to artificial input restoring the impaired senses and computer algorithms translating information from one sense to another. However, while people automatically recognise information processed through their natural crossmodal correspondences (see our blog for the crossmodal correspondences between the senses), they have to learn to learn to interpret the sensations from both brain implants for hearing and sensory substitution devices from vision to hearing (see our blog for A Feel for Art.)

I have invited Carina Sabourin, Yaser Merrikhi, and Stephen G. Lomber, Cerebral Systems Laboratory, McGill University to write this blog post about crossmodal brain plasticity and empowering of sensory abilities. Carina Sabourin, Yaser Merrikhi, and Stephen G. Lomber investigate cortical plasticity in the auditory and visual cortices following hearing loss and the initiation of hearing with cochlear prosthetics. And, recently, in a comprehensive review study, they addressed the question “Do the blind hear better?”.

The idea that blind people can compensate for their lack of vision with enhanced hearing or other abilities, has been around for millennia. Many of the most acclaimed artists from the 8th century BC Greek poet, Homer, to the great jazz musician, Stevie Wonder, lost their vision. Recently, researchers have investigated these anecdotes and confirmed that there is now over-whelming evidence the blind have specific super hearing abilities compared to the sighted1. More excitingly, the brains of blind individuals recruit neural areas that typically handle vision to process auditory information along with hearing brain areas. The ability of typically visual areas to adapt to auditory input is called crossmodal plasticity. The extra brain power crossmodal plasticity provides gives blind individuals their superhuman hearing abilities.

Crossmodal plasticity can occur for other senses beyond hearing too. Deaf individuals recruit hearing brain areas to improve their vision2. Beyond other senses taking advantage of the freed-up brain power, brain plasticity can help the brain adapt to artificial input from brain implants restoring the lost sense. One example is cochlear implants which bypass the inner ear and directly stimulate the auditory nerve giving people with certain types of hearing loss access to sounds. Crossmodal plasticity is thought to help visual and hearing brain areas work together to better process speech. The more teamwork between visual and hearing brain areas, the better cochlear implant users can understand speech3. Similarly, researchers and engineers developing tools for blind people can leverage brain plasticity as well as the specific super hearing abilities of the blind.

One such attempt is sensory substitution devices (SSD) which translate information from one sensory modality into another. Audio-to-visual SSDs convert visual scenes captured by a camera into soundscapes. These devices exploit the improved pitch discrimination4-5 and sound localization abilities6-8 of blind people to convey information about visual environments as the frequency and movement of sounds. SSDs can even use the available brain space in the visual cortex. The part of the visual cortex that recognizes human bodies and tracks their movement was recruited to localize body movement conveyed by an SSD9. The visual reading brain area was even activated by SSDs to enable blind individuals to read with sounds10. Even years after getting their vision back, the visual cortex of individuals who sight was restored through gene therapy was still helping hearing brain areas process sounds11. Some concern exists that crossmodal plasticity may hinder sight restoration from visual brain implants. However, brain plasticity may help the visual and hearing brain areas work together to improve vision outcomes for visual brain implant users, just like it improved the ability of cochlear implant users to understand speech3.

The additional brain power provided by crossmodal plasticity empowers blind individuals with their extraordinary hearing abilities. Researchers and engineers creating tools for the blind can leverage both brain plasticity and their remarkable auditory skills to improve how blind individuals navigate and interact with the world around them.

See our blog for Activities; especially 22-24.

Some suggestions for further listening and watching:

Healing the brain via multisensory technologies and using these to better understand the brain

Losing and recovering sight

Neuroplasticity Animation

The Brain

What Does Blindness or Deafness Tell Us About Brain Development?

What is the function of auditory cortex when it develops in the absence of acoustic input?

 

_______________

1Sabourin, C. J., Merrikhi, Y., & Lomber, S. G. (2022). Do blind people hear better? Trends in Cognitive Sciences, 26(11), 999-1012. https://doi.org/10.1016/j.tics.2022.08.016

2Bavelier, D., Dye, M. W. G., & Hauser, P. C. (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512-518. https://doi.org/10.1016/j.tics.2006.09.006

3Anderson, C. A., Wiggins, I. M., Kitterick, P. T., & Hartley, D. E. H. (2017). Adaptive benefit of cross-modal plasticity following cochlear implantation in deaf adults. Proceedings of the National Academy of Sciences of the United States of America, 114(38), 10256-10261. https://doi.org/10.1073/pnas.1704785114

4Collignon, O., Dormal, G., Albouy, G., Vandewalle, G., Voss, P., Phillips, C., & Lepore, F. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. Brain, 136(9), 2769-2783. https://doi.org/10.1093/brain/awt176

5Rokem, A., & Ahissar, M. (2009). Interactions of cognitive and auditory abilities in congenitally blind individuals. Neuropsychologia, 47(3), 843-848. https://doi.org/10.1016/j.neuropsychologia.2008.12.017

6Chen, Q., Zhang, M., & Zhou, X. (2006). Spatial and nonspatial peripheral auditory processing in congenitally blind people. NeuroReport, 17(13), 1449-1452. https://doi.org/10.1097/01.wnr.0000233103.51149.52

7Lewald, J. (2013). Exceptional ability of blind humans to hear sound motion: Implications for the emergence of auditory space. Neuropsychologia, 51(1), 181-186.

https://doi.org/10.1016/j.neuropsychologia.2012.11.017

8Röder, B., Teder-Salejarvi, W., Sterr, A., Rosler, F., Hillyard, S. A., & Neville, H. J. (1999). Improved auditory spatial tuning in blind humans. Nature, 400(6740), 163-166.

9Striem-Amit, E., & Amedi, A. (2014). Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People “Seeing” by Using Sounds. Current Biology, 24(6), 687-692. https://doi.org/10.1016/j.cub.2014.02.010

10Striem-Amit, E., Cohen, L., Dehaene, S., & Amedi, A. (2012). Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind. Neuron, 76(3), 640-652. https://doi.org/10.1016/j.neuron.2012.08.026

11Mowad, T. G., Willett, A. E., Mahmoudian, M., Lipin, M., Heinecke, A., Maguire, A. M., Bennett, J., & Ashtari, M. (2020). Compensatory Cross-Modal Plasticity Persists After Sight Restoration. Frontiers in Neuroscience, 14(12 May). https://doi.org/10.3389/fnins.2020.00291

Sensory mismatch

tgraven/ / Hearing, Posts, Sensory mismatch, Vision

Even though S.B. had regained his vision, after more than fifty years of blindness, he always closed his eyes in traffic. He continued to function as if he was still blind.

But why? Does information about traffic not transfer from one sense to another? (See our blog for the scientific approach and the crossmodal correspondences between the senses.)

Without vision, S.B. would be used to listening for human activity and unique spatial characteristics when identifying an area, such as conversations about fish and the openness of the sea. He would be helped by echoes of his own sounds and sounds that stand out from the background, like a fan in a quiet street. (See how The Oxford Sound Album started.) S.B. would have listened for contrasting sounds. And enjoy echoes in narrow places; Walking up Queen’s Lane and New College Lane, constantly varying soft sounds; on Radcliffe Square on an early summer morning, and quiet but informative places; Walking inside the University Church of St. Mary the Virgin. Nonetheless, he may have been inspired by the chaotic sounds while Walking through Clarendon Centre. Rain and snow mask information: making it hard, for example, to determine the speed of cars and the distance to a building. To be able to navigate in chaotic and masked sounds, S.B. would have had to tap harder with his white cane to create a better echo against walls, bushes, and poles at pedestrian crossings. But the rain would also help him hear silent objects, like parked bikes and street furniture.

S.B. was experienced in listening for, paying attention to, and intervening with sounds in traffic, and he would have added tactile information too (typically from his white cane, face, and feet) about surfaces, obstructions, and gradients. So, what changes with vision?

Had S.B. started using vision alone, then he would constantly be looking for shapes, for example, in the city skyline, colours, and textures on the Royal Mile. He would have enjoyed how the sunlight enhanced and changed the shapes. As well as three-dimensional architecture and movement, like bay windows and passersby. He would have received information about human activities and danger from signs only.

Indeed, using vision alone, S.B. would be looking for and paying attention to some of the same information about traffic as when he was blind, namely shapes and surfaces/textures; however, not intervene with it. So, again, why did S.B. after having regained vision continue to function in traffic as when he was blind?

There seem to be two possible explanations. One is that when people combine vision and hearing, paying attention to visual shapes reduces their attention to sound and vice versa. S.B. then, who was so very experienced in paying attention to the sounds in traffic, would not have focused on any visual information regardless of whether his eyes were open or closed. Perhaps not even knowing they were closed. Or perhaps getting so confused by the visual information that he closed them. The other explanation is that when people combine visual and auditory information, they tend not to pay attention to either unless the two match. For S.B. a match may never have been possible: he was too experienced in hearing.

Now challenging and inspiring us to use hearing alone, vision alone, and both senses together:

Hearing Princes Street – St David Street

Seeing Princes Street – St David Street

Hearing and seeing Princes Street – St David Street

 

Hearing inside a coffee shop

Seeing inside a coffee shop

Hearing and seeing inside a coffee shop

 

Hearing Princes Street

Seeing Princes Street

Hearing and seeing Princes Street

 

Hearing Victoria Street

Seeing Victoria Street

Hearing and seeing Victoria Street

See our blog for Activities; especially 13-15.

         Some suggestions for further listening and reading:

Herssens, J., Roelants, L., Rychtáriková, M., & Heylighen, A. (2011). Listening in the Absence of Sight: The Sound of Inclusive Environments. Proceedings of Include 2011, 10-17.

Sabourine, C. J., Merrikhi, Y., & Lomber, S. G. (2022). Do blind people hear better? Trends in Cognitive Sciences, 26(11), 999-1012.

Schafer, R. M. (1994). The Soundscape: Our Sonic Environment and the Tuning of the World. Rochester, Vermont: Destiny Books.

Southworth, M. (1969). The Sonic Environment of Cities. Environment and Behavior, 1(1), 49-70.

Stein, B. E., & Meredith, M. A. (1993). The merging of the senses. Cambridge: The MIT Press.

The Oxford Sound Album

The World Soundscape Project