Emotional Resonance – A Multi-Sensory Interactive Experience of Sound and Vision
An exploration of the ways in which humans perceive, empathize and communicate in this 20-minute learning process.
Core Idea:
To explore how contemporary art can redefine the expression and understanding of human emotions through interactive and multi-sensory experiences that connect sound and image to human emotions.
Total Time: 20 minutes
Required Materials:
– A computer or tablet device that supports keyboard input.
– Stable internet connection to ensure smooth page loading and responsive input（need to use the website 帕塔普 (patatap.com)).
– Any tool that can be used for drawing (such as watercolour pencils, coloured pencils, crayons, etc., but also electronic devices such as memos, procreate, etc.)
Activity Steps:
-
Introduction: the dissolution of the border between technology and art (5 minutes)
-
Kandinsky’s visual language
Use colorful combinations of geometric shapes to express the rhythmic sense of music
Kandinsky, a pioneer of abstract painting in the early 20th century, believed in the “resonance” between sound and image. By expressing tones through colors and rhythms through shapes, he regarded visual art as a language that touches emotions and is connected to music.
-
Interaction between sound and image.
– Question: “If sound and image can interact with each other as Kandinsky understood, how can we use technology today to realize this emotional resonance?”
-
Exploration and Experience: Triggers of Perception and Empathy (7 minutes)
– By listening to 3-5 different sound clips at the push of a button, learners can perceive different emotions (e.g., pleasure, anxiety, serenity)
Audio tracks from 帕塔普 (patatap.com)). , licensed for reuse as per platform policies.
– Use words record the emotions you feel in different sound clips.
- What scene or emotion does this sound remind you of?
- How does your graphic and emotional expression correlate?
– Draw your feelings about the sound, using geometric shapes of different colors to illustrate their own unique emotions about the sound.
Examples of geometric shapes.
Mapping demo video (memo example)
This video is an original creation by Chuni Mao. All rights reserved.
-Experience the interactivity of Patatap (帕塔普 (patatap.com)).You can use the trackpad to tap the screen or press the letters A-Z.
Example video of using patatap
This video is an original creation by Chuni Mao. All rights reserved.
-
Reflection and discussion: co-creation in contemporary art (5 minutes)
– Have learners share emotional clips of creation, discuss what they found in the combination of sound and image, and reflect:
- Would different people have similar or different experiences in the same sound clip? Why？
- Does this interactive experience have the power to express emotions across languages and cultures? Why？
– Post your answers in the comments section and read others’ thoughts and interact with other comments!
Emotional Resonance – A Multi-Sensory Interactive Experience of Sound and Vision © 2024 by Chuni Mao is licensed under CC BY-SA 4.0 
(Toolkit cover © 2024 by Chuni Mao is licensed under CC BY-SA 4.0 )
(Use colorful combinations of geometric shapes to express the rhythmic sense of music. © 2024 by Chuni Mao is licensed under CC BY-SA 4.0 )
(Examples of geometric shapes © 2024 by Chuni Mao is licensed under CC BY-SA 4.0 )
5 replies to “Emotional Resonance – A Multi-Sensory Interactive Experience of Sound and Vision”
Leave a reply
You must be logged in to post a comment.
I think different people experience the same sound clip differently and it has the power to express emotions across languages and cultures.
People may not have the same associations to the sounds, but the way that emotion translates over cultures may be possible. I associate greens and blues with relaxation, this has been sturdier to be relatively consistent across culture. Additionally, sounds that are understood to be softer are associated with rounder shapes (the bouba and Kiki effect) so it is possible that people would have similar associations
Because I think different rhythms and tones of sound give me different sensations, slow and low sounds give me the feeling of mountains with curves, calm and low saturated tones, while sharp sounds give me sharpness and hardness, bright colours
1
When I hear these sounds, different kinds of emotions arise within me, and I try to weave them together, through different colours and lines.