Category: Crossmodal brain plasticity

Reading braille in colour

tgraven/ / Colour, Crossmodal brain plasticity, Crossmodal correspondences, Haptic touch, Multisensory, Vision

Previous research has found that people who have just started reading braille by haptic touch try to imagine the characters visually. For example, seeing black spots on a white background and associating the formation of these with regular print characters, objects they have seen, or both. When they become tactually more experienced, this stops. In contrast, people who are born blind recognise the braille characters through either their quantity and location of dots or their tactile global shape1. (See our blog for the scientific approach, Vision, haptic touch, and hearing, and Decay and maintenance of sensory memories.)

This time, I wanted to explore what happens when people stop trying to imagine the braille characters visually. To shed some light on this, I have invited I.A. to share her experiences. Born partially sighted, with about 5% vision in one eye, I.A. started reading and writing regular print. I.A. learned braille on her own at around 13 years old: first memorising the braille alphabet, using a combination of vision and haptic touch. Then reading materials published in both regular print and braille. About three years later, I.A. was no longer able to read regular print. She now knew all the braille characters by haptic touch, but needed help to perfect her reading technique. Today, I.A. has been reading and writing braille by haptic touch alone for more than 25 years. She has been teaching braille to people who have become blind for more than 10 years. And she is an appointed Board Member of the braille authority in her country. I.A. approved this text before we posted it on our blog. The journey she is taking us on is rather unexpected.

For the first two or three years of reading braille by haptic touch alone, I.A. saw the dots in each character as mini light bulbs in her mind’s eye. She concentrated on how many they were and where they were located.

Gradually, I.A. started perceiving the tactile global shape of short two and three letter words. For example, of ⠉⠁⠞ (cat) and ⠙⠕⠛ (dog). She still reads longer words letter by letter, but she has stopped seeing their dots as mini light bulbs in her mind’s eye.

Reading by haptic touch alone, I.A. recognises the braille characters through their quantity and location of dots. She recognises short two and three letter words by their tactile global shape.

As I.A. became more experienced in reading braille by haptic touch alone, the letters started appearing in colour – just as they had done in regular print. For example, a in red; b in dark blue, c in light yellow, d in dark yellow, e in pale blue; f in blue-grey; g in green; h in beige; i in translucent white, j in white, and so on.

And the numbers too: 1 in white; 2 in yellow; 3 in blue, 4 in light yellow; 5 in green; 6 in blue, 7 in white, 8 in red, 9 in brown, and 0 in silver grey. Some punctuations and signs also have the same colour as in regular print; like “division” (÷) which is yellow, while others gained a colour in braille, like “equals” (=) which is now mossy green.

I.A. experiences synaesthesia. She associates certain letters, numbers, punctuations, and signs with certain colours (see our blog for the crossmodal correspondences between the senses and the intriguing association between sounds and colours).

I.A.’s synaesthesia appears only when reading. She does not associate letters, numbers, punctuations, and signs with colour when writing. But, as soon as she checks her spelling, they appear in colour again.

When reading shorter words, like “cat” and “dog”, I.A. first detects the tactile global shape of the word and then associates it with colour. That is, the colour when combining all the letters. The word “dog” for example is a combination of (d) dark yellow, (o) silver grey, and (g) green. I.A. sees “dog” as a mossy green word.

According to I.A. the colour helps her distinguish between words with a similar tactile global shape. For example, ⠙⠕⠛ (dog) and ⠋⠕⠛ (fog). While “dog” has a mossy green colour, “fog” is light green; that is, a combination of (f) blue-grey, (o) silver grey, and (g) green.

Longer words, that I.A. reads letter by letter, are coloured by their first letter. For example, the word “braille” is dark blue and “crossmodal” is light yellow.

It seems I.A.’s subjective crossmodal correspondences are not sensory specific: they exist regardless of whether she is reading regular print or braille. Could they be linked to information that transfers between vision and touch? For example, shape. Or, are they rather linked to the sound of the letters, numbers, punctuations, and signs (see our blog for the scientific approach, the crossmodal correspondences between the senses and the intriguing association between sounds and colours)?

See our blog for Activities; especially 58-61.

_______________

1Graven, T. (2015). How blind individuals discriminate braille characters: An identification and comparison of three discrimination strategies. British Journal of Visual Impairment, 33(2), 80-95.

Visual memories and sensory experiences

tgraven/ / Crossmodal brain plasticity, Crossmodal correspondences, Haptic touch, Hearing, Multisensory, Posts, Smell, Taste, Vision

It seems people who lose vision use information that transfers between the senses to retrieve visual memories. And that their visual memories decay when they have reached a certain level of experience in the other senses. (See our blog for the scientific approach, the crossmodal correspondences between the senses, and Decay and maintenance of sensory memories). But what happens when people who are born blind gain vision and then lose it again?

I have invited P.B. to share his experiences of visual memories and sensory information when going from blind to partially sighted and back to being blind again. P.B. was born blind with some light perception in one eye. After surgery and other medical treatments, he had gained about 7% vision in the other eye at three and a half years old. P.B. does not remember this shift from being blind to being partially sighted. Then, in his early twenties, P.B. had an accident that rendered him totally blind in both eyes. P.B. thinks of himself as a sighted person who happens not to see. He visualises when making plans. And his dreams are always visual. P.B. approved this text before we posted it on our blog.

P.B. has several visual memories from when he was little. From the house he grew up in, for example, he remembers the light from a chandelier hanging over the coffee table in the living room, the patterned wallpaper in his room, and the countries on the world map hanging on the wall. And he remembers the colours on the outside walls, the doors, and the window frames.

Today, when he has no new visual input, P.B. creates visual images based on his memories and knowledge. For example, that of a White Swiss Shepherd Dog working as a guide dog from his memories of both the white colour and the German Shepherd Dog. As well as his knowledge of guide dogs.

His visual memories are typically triggered by somebody describing how something looks, like a bright red sunset – not by information from the other senses.

P.B.’s visual memories “flash up almost like the flavour when eating or drinking”.

He has to decide not to focus on visual memories and images when talking to people: the memories are now 20 years old and people have changed. And his created visual images of what new people look like may be very wrong.

P.B. has to actively suppress his urge to retrieve visual memories or create visual images of people based on their voices.

He describes not having visual memories and images of spatial relations and distances. P.B. rather remembers them through his body. He walks around in the city centre with no vision, a podcast or some music in his ears, and shoes with a thick curved sole (which makes it difficult to feel the surfaces on pavements, streets, steps, floors, etc.). He does not count steps or any of that and pauses the podcast or music in his ears only when he feels he might get lost.

For P.B., sensory experiences include a variety of simultaneous sensations as well as visual memories and created images – they are multisensory. The forest, for example, is the sound of wind and trees, the texture of surfaces under his feet, scents, and the memories of colours. He zooms in on whatever attracts his attention: indeed, not on one piece of sensory information after the other in a certain order. For example, zooming in on a sudden shift from soft to hard texture under his feet.

An interesting sensory experience, according to P.B., is that of new flavours, the texture of the food, and the sound of crunch when chewing, He describes a cloudberry cream dessert as “not very sweet, but also not sour, light orange colour, and creamy texture”. But a redcurrant jelly only as “something wobbling on the plate”. He does not describe the flavour, scent, and/or feel in his mouth – merely how the jelly looks on the plate.

Was he not able to suppress his visual memories of jelly – did they take over?

When P.B. is sensory tired, he relaxes his senses by listening to “concentration music, like opera or prog rock where I can immerse myself and get absorbed by a different universe”.

It seems P.B. can maintain his visual memories and ability to create visual images through people’s descriptions of what something looks like. But, at the same time, these memories and images are distracting him from focusing on sensory information that is more relevant to him today with no vision, for example, people’s voices and the feel of wobbling jelly in his mouth, So is this, in fact, a negative spiral – that maintaining visual memories prevents experiences in the other senses, which in turn helps nurture the visual memories?

See our blog for Activities; especially 49-51.