Author: tgraven

Crossmodal brain plasticity and empowering of sensory abilities

tgraven/ / Crossmodal brain plasticity, Crossmodal correspondences, Hearing, Posts, Vision

Research on crossmodal brain plasticity has not only found that the brain compensates for sensory impairments. For example, so that people who are born blind process auditory information in both the auditory and the visual areas – not just the auditory like the fully sighted. (See our blog for the scientific approach.) It has also shown that the brain adapts to artificial input restoring the impaired senses and computer algorithms translating information from one sense to another. However, while people automatically recognise information processed through their natural crossmodal correspondences (see our blog for the crossmodal correspondences between the senses), they have to learn to learn to interpret the sensations from both brain implants for hearing and sensory substitution devices from vision to hearing (see our blog for A Feel for Art.)

I have invited Carina Sabourin, Yaser Merrikhi, and Stephen G. Lomber, Cerebral Systems Laboratory, McGill University to write this blog post about crossmodal brain plasticity and empowering of sensory abilities. Carina Sabourin, Yaser Merrikhi, and Stephen G. Lomber investigate cortical plasticity in the auditory and visual cortices following hearing loss and the initiation of hearing with cochlear prosthetics. And, recently, in a comprehensive review study, they addressed the question “Do the blind hear better?”.

The idea that blind people can compensate for their lack of vision with enhanced hearing or other abilities, has been around for millennia. Many of the most acclaimed artists from the 8th century BC Greek poet, Homer, to the great jazz musician, Stevie Wonder, lost their vision. Recently, researchers have investigated these anecdotes and confirmed that there is now over-whelming evidence the blind have specific super hearing abilities compared to the sighted1. More excitingly, the brains of blind individuals recruit neural areas that typically handle vision to process auditory information along with hearing brain areas. The ability of typically visual areas to adapt to auditory input is called crossmodal plasticity. The extra brain power crossmodal plasticity provides gives blind individuals their superhuman hearing abilities.

Crossmodal plasticity can occur for other senses beyond hearing too. Deaf individuals recruit hearing brain areas to improve their vision2. Beyond other senses taking advantage of the freed-up brain power, brain plasticity can help the brain adapt to artificial input from brain implants restoring the lost sense. One example is cochlear implants which bypass the inner ear and directly stimulate the auditory nerve giving people with certain types of hearing loss access to sounds. Crossmodal plasticity is thought to help visual and hearing brain areas work together to better process speech. The more teamwork between visual and hearing brain areas, the better cochlear implant users can understand speech3. Similarly, researchers and engineers developing tools for blind people can leverage brain plasticity as well as the specific super hearing abilities of the blind.

One such attempt is sensory substitution devices (SSD) which translate information from one sensory modality into another. Audio-to-visual SSDs convert visual scenes captured by a camera into soundscapes. These devices exploit the improved pitch discrimination4-5 and sound localization abilities6-8 of blind people to convey information about visual environments as the frequency and movement of sounds. SSDs can even use the available brain space in the visual cortex. The part of the visual cortex that recognizes human bodies and tracks their movement was recruited to localize body movement conveyed by an SSD9. The visual reading brain area was even activated by SSDs to enable blind individuals to read with sounds10. Even years after getting their vision back, the visual cortex of individuals who sight was restored through gene therapy was still helping hearing brain areas process sounds11. Some concern exists that crossmodal plasticity may hinder sight restoration from visual brain implants. However, brain plasticity may help the visual and hearing brain areas work together to improve vision outcomes for visual brain implant users, just like it improved the ability of cochlear implant users to understand speech3.

The additional brain power provided by crossmodal plasticity empowers blind individuals with their extraordinary hearing abilities. Researchers and engineers creating tools for the blind can leverage both brain plasticity and their remarkable auditory skills to improve how blind individuals navigate and interact with the world around them.

See our blog for Activities; especially 22-24.

Some suggestions for further listening and watching:

Healing the brain via multisensory technologies and using these to better understand the brain

Losing and recovering sight

Neuroplasticity Animation

The Brain

What Does Blindness or Deafness Tell Us About Brain Development?

What is the function of auditory cortex when it develops in the absence of acoustic input?

 

_______________

1Sabourin, C. J., Merrikhi, Y., & Lomber, S. G. (2022). Do blind people hear better? Trends in Cognitive Sciences, 26(11), 999-1012. https://doi.org/10.1016/j.tics.2022.08.016

2Bavelier, D., Dye, M. W. G., & Hauser, P. C. (2006). Do deaf individuals see better? Trends in Cognitive Sciences, 10(11), 512-518. https://doi.org/10.1016/j.tics.2006.09.006

3Anderson, C. A., Wiggins, I. M., Kitterick, P. T., & Hartley, D. E. H. (2017). Adaptive benefit of cross-modal plasticity following cochlear implantation in deaf adults. Proceedings of the National Academy of Sciences of the United States of America, 114(38), 10256-10261. https://doi.org/10.1073/pnas.1704785114

4Collignon, O., Dormal, G., Albouy, G., Vandewalle, G., Voss, P., Phillips, C., & Lepore, F. (2013). Impact of blindness onset on the functional organization and the connectivity of the occipital cortex. Brain, 136(9), 2769-2783. https://doi.org/10.1093/brain/awt176

5Rokem, A., & Ahissar, M. (2009). Interactions of cognitive and auditory abilities in congenitally blind individuals. Neuropsychologia, 47(3), 843-848. https://doi.org/10.1016/j.neuropsychologia.2008.12.017

6Chen, Q., Zhang, M., & Zhou, X. (2006). Spatial and nonspatial peripheral auditory processing in congenitally blind people. NeuroReport, 17(13), 1449-1452. https://doi.org/10.1097/01.wnr.0000233103.51149.52

7Lewald, J. (2013). Exceptional ability of blind humans to hear sound motion: Implications for the emergence of auditory space. Neuropsychologia, 51(1), 181-186.

https://doi.org/10.1016/j.neuropsychologia.2012.11.017

8Röder, B., Teder-Salejarvi, W., Sterr, A., Rosler, F., Hillyard, S. A., & Neville, H. J. (1999). Improved auditory spatial tuning in blind humans. Nature, 400(6740), 163-166.

9Striem-Amit, E., & Amedi, A. (2014). Visual Cortex Extrastriate Body-Selective Area Activation in Congenitally Blind People “Seeing” by Using Sounds. Current Biology, 24(6), 687-692. https://doi.org/10.1016/j.cub.2014.02.010

10Striem-Amit, E., Cohen, L., Dehaene, S., & Amedi, A. (2012). Reading with Sounds: Sensory Substitution Selectively Activates the Visual Word Form Area in the Blind. Neuron, 76(3), 640-652. https://doi.org/10.1016/j.neuron.2012.08.026

11Mowad, T. G., Willett, A. E., Mahmoudian, M., Lipin, M., Heinecke, A., Maguire, A. M., Bennett, J., & Ashtari, M. (2020). Compensatory Cross-Modal Plasticity Persists After Sight Restoration. Frontiers in Neuroscience, 14(12 May). https://doi.org/10.3389/fnins.2020.00291

A Feel for Art

tgraven/ / Art, Haptic touch, Posts, Uncategorised

There is more to touching art than merely using the hands and fingers to recognise depictions of landscapes and victorious kings on horses; ornate chairs and tables with carved and inlaid decorations; geometric shapes and stylised forms of flowers; figurative and abstract fruits and animals. (See our blog on Vision, haptic touch, and hearing.)

In this blog post, I have invited Professor Georgina Kleege, University of California, Berkeley to write about touching art: why she likes to do it, how she does it, and why she thinks others would get something out of the experience. Georgina Kleege has published numerous books and papers, and been awarded the Distinguished Teaching Award twice. She is an art lover and is well known for advocating for more tactile opportunities for all museum visitors, including those who are fully sighted.

As a blind person, I enjoy the incredible privilege of touch access to art works at museums around the world who offer such opportunities. Over the years I have accumulated many observations and insights that inspire me to dispel common myths and misconceptions about touch perception, specifically in an artistic context. For one thing, I find that the notion of sensory translation or substitution can be misleading. People seem to want to make an analogy between the two eyes of the sighted and the two hands of the blind, as if simply laying a hand on the art object will summon a detailed image to the blind person’s mind’s eye. There’s a mistaken impression that the point of touching the art object is merely to determine what it’s representing visually—what it looks like to people who can see. This implies that sighted people don’t need to touch the art because they can see it, when in fact, determining what the art object might be depicting is not always, I’d say rarely, the most interesting thing about the experience.

When touch is merely an exercise to identify a shape, one uses a minimum of the tactile apparatus available. One traces the outline of the object in a manner analogous to the way the visual system separates the object from its background. One uses mainly the fingertips which also deliver some information about the surface texture and temperature. They can also discover fine details in carving, or the seams, joints and welds that hold the thing together, and even signs of past damage and repairs which may not be available to the eyes alone. But to grasp the objects three dimensionality one must grasp: wrap one’s fingers and palms around volumes, drape the whole hand around contours. The action of the hands and the skin of the palms delivers more information. The movement of the hands inspires other movement, of the whole arm, of the spine, as one reaches, stretches, bends and extends to take in the form in its entirety. There’s no point sticking to one place—the vantage point for sighted people. One is better off moving around doing what I have come to call dancing with the sculpture, circumnavigating the object while maintaining light contact with one hand. This action can convey a sense of composition, of symmetry and dynamism. These techniques work equally well whether the sculpture is figurative or abstract.

I used to avoid referencing my emotional response to touching art. English, like other languages, conflates touch sensation with the emotions. We find a work of art touching and it makes us feel happy or sad. This conflation is problematic for blind people who rely more on touch than our sighted peers because touch is often considered to be a lesser way of knowing the world, more animalistic or infantile. Babies rely on touch before their visual perception fully develops. But now I lean into this connection. I actively scan my emotions for a response to what I’m touching. Touch can be intuitional; the sculpture tells me how to touch it. It reveals aspects of itself sequentially, as I experiment with different methods and repeat or reject actions according to what feels most generative. It is an accretive process requiring attention, sensitivity sometimes even playfulness. And the emotions this touching summons in me may have something to do with what the artist hoped to convey. If nothing else, my hands and body replicate the artist’s own gestures and movements which in turn may link me to the ideas and emotions that went into the sculpture’s creation.

Recently, I’ve been advocating for more tactile opportunities for all museum visitors, including sighted people who are typically excluded from this form of access. For now, conservation and crowd management concerns make it unlikely that everyone will be allowed to get their hands on art. So instead, I endeavor to describe my experiences, as here, in the hope that sighted art lovers can profit, if only vicariously. (See also Nydia, the Blind Flower Girl of Pompeii.)

See our blog for Activities; especially 19-21.

Some suggestions for further listening, reading, and watching:

A Conversation about Blindness and Art

Art Beyond Sight

Best Things To Touch As A Blind Person

Do Touch The Artwork At Prado’s Exhibit For the Blind

Some Touching Thoughts and wishful Thinking

The Gravity, The Levity: Let Us Speak of Tactile Encounters

Touch and See

Worst Things To Touch As A Blind Person