Decay and maintenance of sensory memories

Previous research has found that memories based mainly on sensory information decay if they are not maintained. For example, people who have become blind are likely over time to lose their visual memories and, thus, the ability to visually imagine objects, shapes, and faces. However, people who have lost hearing seem to have memories of both sounds and voices.


This time I present some lived experiences shared by people who have become blind.


Often people who have just started learning braille by touch try to imagine the characters visually. For example, seeing “black spots on a white background” and then associating:

– the braille o (⠕) with the print close parenthesis

– the braille v (⠧) with the print capital letter L

– the braille s (⠎) with a snake

– the braille t (⠞) with a chair or set of stairs in profile1-2


They associate braille characters either with print characters; through focusing on angles, curves, and straight lines, or with object shapes. But this all stops when they become tactually more experienced: “I associated braille characters with regular print characters in the beginning (…). Not now”2.


(…), he could perfectly well, visually, imagine a painting hanging over his living-room sofa, but could no longer, visually, imagine his wife’s, his daughter’s, or even his own face: these had now become tactually familiar3.

So intense was my desire to know the face of a stranger, as someone I was meeting for the first time, that vivid pictures of the person’s possible face would flash through my memory so rapidly that I could hardly concentrate on what they were saying. Slowly, slowly, that also began to fade. (…) I began to lose the memory that things looked like anything. I found myself caught with a slightly abrupt sense of surprise when people would say to me, “John, would you like to know what I look like?”4


And then, when the other senses had taken over, John Hull described it as “being reborn”.


I discovered so many beautiful things. For example, trees came back. I used to love trees – the forest, the greenery. Now stars had gone. Clouds had gone. The horizon was no more. But now I gradually discovered trees came back. They came back acoustically. (…) I discovered that in the winter, the trees whistled, and cracked, and hissed. In the spring, they became all fluffy. In the summer, they were like the rolling ocean waves as the wind swept across them. In the autumn, they became all tinkly. (…) And I felt, how incredibly beautiful that is. Why did I never notice it before?4


It seems people who lose vision use information that transfers between the senses to retrieve visual memories and, thus, visually imagine the world around them; that is, until reaching a certain level of experience in the other senses. And; at that point, their visual memories are gone. (See our blog for the scientific approach and the crossmodal correspondences between the senses.) In contrast, it seems people who lose hearing remember both sounds and voices. Could this be because they previously perceived multisensory information, for example, lip-reading by vision and voice by hearing (sound on), and that visual information maintains their auditory memories and, thus, their ability to imagine the auditory world around them?

 

See our blog for Activities; especially 31-33.

_______________

1Graven, T. (2018). How individuals who are blind locate targets. British Journal of Visual Impairment, 36(1), 57-74.

2Graven, T. (2015). How blind individuals discriminate braille characters: An identification and comparison of three discrimination strategies. British Journal of Visual Impairment, 33(2), 80-95.

3Graven, T. (2009). Seeing Through Touch: When Touch Replaces Vision as the Dominant Sense Modality. Saarbrücken: VDM Verlag Dr. Müller AG & Co.

4John Hull Blindness and memory being reborn into a different world

On the intriguing association between sounds and colours

It seems three main types of crossmodal correspondences between the senses exist: transfer of information, shared associations, and subjective associations (see our blog for the crossmodal correspondences between the senses).


In this blog post, I have invited Researcher Nicola Di Stefano, Institute of Cognitive Sciences and Technologies, National Research Council of Italy to explain the subjective associations between music and colour. Nicola Di Stefano has contributed numerous publications on both the philosophy and psychology of perception and the aesthetics and psychology of music.


Sounds and colours are two distinct sensory experiences that convey different information about the environment we inhabit. While we typically attribute a colour to every object we perceive, we wouldn’t assert that each object possesses or is inherently associated with a particular sound. Of course, musical instruments produce sounds, and various objects can emit sounds, like hammers, rocks, and sticks, but sound seems to be an ontologically different, namely less foundational, feature of objects compared to colours.


Interestingly, however, intellectuals, researchers, artists, and composers have been long fascinated by the association between those two seemingly radically different sensory experiences. Their idiosyncratic association is evident in sound-colour synaesthesia, one of the most prevalent forms of synaesthesia, a rare neurological phenomenon where stimulation of one sensory (or cognitive) pathway leads to automatic, involuntary experiences in another1. This cross-wiring allows individuals with synaesthesia to experience a unique blending of sensations, such as seeing colours in response to musical notes or chords. Sound-colour synaesthesia has inspired several artworks, including the musical paintings by Kandinsky and Scriabin’s Prometheus, a composition based on the combination of coloured lights and music.

Coloured lights on a board in Scriabin's house

 




Photo retrieved from: Scriabin and the Possible


Psychologists have also explored the mechanisms underlying the consistent association between certain features of sounds and colours in non-synaesthetes. The concept of “crossmodal correspondence” suggests that certain sensory attributes share underlying perceptual or cognitive processes, leading to associations between them2. For instance, studies have revealed that people tend to associate high-pitched sounds with light or bright colours, while low-pitched sounds are often linked to dark colours3. These associations may arise from shared perceptual features, such as the frequency or intensity of auditory and visual stimuli.


One of the most intuitive ways to explain sound-colour correspondences is psychophysical, suggesting that both sounds and colours are vibratory phenomena. However, the sensory systems that process the two signals are quite different, making it challenging to establish a link between sounds and colours based solely on alleged psychophysical similarity. Additionally, an important distinction lies in the octave similarity in music, where sounds at different frequencies (integer multiples of the same fundamental frequency) share the same pitch class (e.g., “D”), whereas in the domain of colour, there is no equivalent octave repetition.


Furthermore, philosophers grapple with the metaphysical implications of the interplay between music and colour. Music, often described as the “language of the emotions” elicits powerful affective responses in listeners, shaping their emotional landscapes4-5. Similarly, colour possesses symbolic and emotional resonance, evoking mood and atmosphere in visual art and design. The intentional combination of music and colour in multimedia art forms, such as film and digital media, underscores the transformative potential of blending sensory modalities to create immersive experiences.


Whether through the lens of synaesthesia, crossmodal correspondence, or aesthetic inquiry, the convergence of music and colour illuminates the intricate interplay between sensory perception, cognition, and emotion. By unravelling the mysteries of this symbiotic relationship, researchers, artists, and practitioners aim to gain deeper insights into the nature of human experience and the profound ways in which art shapes our understanding of the world.


See our blog for Activities; especially 25-27.


Some suggestions for further listening and watching:

Artists use synesthesia to expand their creative limits

Elements of Music

Introduction to Color

Is Your Red The Same as My Red?

Light Organ (Clavière a lumiére) – Scriabin op 65 no 2

Seeing Sound: How Synesthesia Can Change Our Thinking

Seeing song through the ears of a synesthete

Synesthesia & creating your own score

 

_______________

1Ramachandran, V. S., & Hubbard, E. M. (2001). Synaesthesia–a window into perception, thought and language. Journal of consciousness studies, 8(12), 3-34.

2Spence, C. (2011). Crossmodal correspondences: A tutorial review. Attention, Perception, & Psychophysics, 73(4), 971-995.

3Spence, C., & Di Stefano, N. (2022). Coloured hearing, colour music, colour organs, and the search for perceptually meaningful correspondences between colour and sound. i-Perception, 13(3), https://doi.org/10.1177/20416695221092802

4Cooke, D. (1959). The language of music. London: OUP.

5Juslin, P. N., & Sloboda, J. (2011). Handbook of music and emotion: Theory, research, applications. Oxford: Oxford University Press.