Table of Contents
2. Overview
2.1 Background
2.2 Process
3. Visual
3.2 Art Concept
4. Audio
4.1 Sound Design
4.2 Composition
4.3 Audio Carrier
5. Interaction Design
5.2 Hardware
5.3 Software
6. Setup
6.1 Decoration
6.2 Venue Setting
7. Promo Video
Author: s2328976
Venue_And_Setup
Venue
Our project was held in Atrium. Before audiences entered the room, we affixed a poster to the front door, introducing our project and effectively conveying its theme.
Figure 1: Poster
Atrium provides enough space for a certain number of spectators to walk around. We placed chairs around the front area to prevent individuals from obstructing the projection or inadvertently stepping on cables. This layout clearly delineated designated pathways for audience movement and restricted areas to maintain an immersive viewing experience.
Figure 2: Venue
Ideally, we aimed for a completely darkened space. However, a large window above Atrium meant that natural light would influence the lightness in Atrium, preventing the space from achieving complete darkness throughout the presentation. Thus, to improve our project, we will need to use a large black cloth to block out the light from the top to create an all-black, immersive, deep-sea-like environment.
Figure 3: Venue
Setup
Because our project is an audio-visual installation with surround sound techniques, the used equipment includes:
- 4 Genelec 1030a speakers
- 1 Faderfox DS3
- 1 HD NEC V302H projector
- 1 FireFace UCX
- 1 projection screen
- 5 sensors
Figure 4: Equipment setup
We wrote instructions on the designed box next to each sensor to guide the participant during the presentation.
Figure 5: Equipment setup
Sound_Design
Layer 1: Sea Monsters
The inspiration for the first layer comes from my research towards thalassophobia. Jarrett (2022) argues that there is a significant relationship between thalassophobia and mythology. Ancient and modern myths of deep-sea monsters have impressions on the collective consciousness, influencing the perceptions and fears associated with the deep sea and shaping individuals’ experiences of thalassophobia.
Based on my findings on sea monsters across various cultural myths and legends, it’s evident that they are often described as exceptionally large creatures.
Figure 1. Kraken
Figure 2: Loch Ness Monster
Figure 3: Leviathan
Figure 4: Hydra
To create the sounds for these sea creatures, I applied Mconvolution on self-recorded material to make them sound mysterious and distorted. I focused more on generating low-frequency, creature-like sounds to correspond with the giant size of these sea monsters. The siganificant low frequencies serve to show their enormity and provide a sense of dread. Additionally, I incorporated reverb into the monsters’ sounds to give a sense of the deep, dark expanse of the sea, enhancing the eerie and ominous atmosphere surrounding them.
Figure 5: Using Mconvolution
The sounds of Siren are the high-frequency sounds that were created in this layer. I pitched up a synth violin and applied a reverb to make these sounds more ethereal and mysterious.
Figure 6: Siren
Layer 2: Choas
The sounds in this layer draw inspiration from research on the physical and mental symptoms associated with thalassophobia. Studies, like that by Bence (2023), highlight common experiences such as dizziness and faintness among individuals with thalassophobia. To deliver this sensation, I layered whoosh sounds to evoke a sense of disarray and chaos. Additionally, I reversed some of the whoosh sounds to further enhance the feeling of dizziness. Introducing high-frequency sounds resembling tinnitus adds an additional layer of disorientation, conveying the sense of dizziness experienced by people with thalassophobia.
I also manipulated some noises through Mconvolution to further highlight the chaos.
This layer is crafted based on Craig’s improvisation.
Figure 7: Collaboration with Music
Layer 3: Unkown
In this layer, my inspiration came from the definition of thalassophobia. Thalassophobia is defined as the persistent fear of vast, deep, and often dark bodies of water that feel dangerous (Anwar, 2021). Specifically, thalassophobia describes a person’s fear of the great unknown in the water (Anwar, 2021). I created some hollow sound effects using Mconvolution, and I applied large reverbs to convey the fear of the unknown.
Furthermore, I incorporated recordings of my voice, which I pitched the recording down and layered together to produce a whisper-like, hollow effect.
Figure 8: Recording my voice
Layer 4: Calmness
The last layer is a peaceful state. Therefore, I used Mconvolution to produce some organic sounds, such as wind, chimes and cave with water sounds. These organic soft sounds contribute to conveying a sense of chill and relaxation with the soft high frequency.
Certain sounds were directly recorded in the studio. These include the crisp sounds of ice cubes gently swirling in water, the delicate touch of a necklace, the pouring of rice, and the subtle rustle of pages turning.
Each sound across the various layers has been carefully trimmed to ensure immediate responsiveness to audience interactions, avioding any delays. This editing enhances the immersive quality of the experience, allowing participants to seamlessly engage with the soundscape and maintain a fluid connection between their actions and the auditory feedback they receive.
Find all the sound effects through the link below:
https://drive.google.com/drive/folders/17_9u3E1jYnotABVYvZ3BxPkt3zhn83_p?usp=drive_link
References
Anwar, B. (2021) ‘Thalasopphobia: Fear of the Ocean‘, TalkSpace, 20 August. Available at: https://www.talkspace.com/mental-health/conditions/thalassophobia/#:~:text=The%20thalassophobia%20definition%20is%20pretty,water%20right%20below%20their%20feet (Accessed: 15 April 2024).
Bence, S. (2023) How Do I Know if I Have Thalassophobia?. Available at: https://www.verywellhealth.com/thalassophobia-5093770 (Accessed: 15 April 2024).
Jarrett, C. (2022) ‘Thalassophobia: Everything you need to know’, BBC Science Focus, 16 July. Available at: https://www.sciencefocus.com/the-human-body/thalassophobia (Accessed: 15 April 2024).
Mythicalcreatures.info (2024) Leviathan. Available at: https://mythicalcreatures.info/characters/leviathan/ (Accessed: 15 April 2024).
Mythicalcreatures.info (2024) The Hydra. Available at: https://mythicalcreatures.info/characters/hydra/ (Accessed: 15 April 2024).
Potts, K. (2014) The hunt for the Loch Ness Monster – valid scientific research or a fool’s errand?. Available at: https://www.abdn.ac.uk/news/6504/(Accessed: 15 April 2024).
Salvador, R. B. & Tomotani B. M. (2014) ‘The Kraken: when myth encounters science’. Medievalists.net. Available at: https://www.medievalists.net/2015/01/kraken-myth-encounters-science/ (Accessed: 15 April 2024).
Villamirella (2024) The legend of Siren Molpé. Available at: https://www.villamirella.it/en-blog/myth-sirena-molpe-palinuro (Accessed: 15 April 2024).
Background
Overview
Mathivanan (2017, p. 374) defines presence as the psychological experience of feeling completely enveloped within a virtual environment. Expanding on this concept, our group is inspired to craft an interactive, immersive installation to create an audio-visual virtual environment.
This project, Thalassophobia, is an immersive audiovisual installation that explores the fear of the deep sea. We aim to create an immersive experience using Max, sensors, and surround sound techniques to fully immerse the audience in the depths of fear.
The idea of having Thalassophobia as the theme is inspired by sea-based horror films, such as Jaws (Spielberg, 1975), Deep Blue Sea (Harlin, 1999), and Open Water (Kentis, 2003). Our project has four different layers. Throughout the immersive journey, the visuals and soundscape will gradually expand, unveiling different layers of Thalassophobia into the experience.
Our original plan was to structure the experience linearly, guiding audiences through escalating levels of fear toward eventual calmness. However, our project took a different approach in the final presentation, opting for a non-linear looping structure that allows participants to shape their experience through the depths of Thalassophobia. We designed four distinct levels, each presenting a different progressive level of phobia using sounds and visuals.
Participants will navigate these levels by actively engaging with sensors and a level controller, allowing for a dynamic and interactive experience. The first level explores the realm of sea monsters, providing a concrete and general understanding of Thalassophobia. The second level explores the physical and mental responses triggered by Thalassophobia, offering insight into its internal impact.
The journey then deepens with the third level, confronting participants with the abstract fear of the unknown inherent in Thalassophobia. Finally, the fourth level offers a peaceful state, reflecting upon and contrasting the deep-seated fears of the ocean’s depths. This four-level loop allows participants to choose their starting and ending points, allowing for a more personalized and creative journey into the depths of Thalassophobia.
Through dynamic audiovisual elements and interactive storytelling, Thalassaphobia aims to evoke a range of emotions and sensations, allowing participants to engage with their creations in an immersive way.
Similar Art Work
PlayLoop
PlayLoop (Wong, 2023) is an innovative interactive installation merging sound and light, allowing participants to craft distinct audio compositions through physical interaction with multiple audio loops within an immersive soundscape. Employing light-emitting diodes and motion-activated sensors, this project offers a dynamic platform for interaction and creativity.
The work embodies two core concepts. Firstly, the most important idea of this project is ‘Group Creation'(Wong, 2023). The installation is designed for multiple participants. The more individuals engage with it, the richer and more complex the sounds become. This installation encourages collaboration, inviting participants to collectively shape the auditory experience through harmonious melodies, discordant rhythms, minimalist arrangements, or complex soundscapes. The second concept of this project is ‘Co-creation’. Participants are encouraged to collaborate with the artist to create soundscape together. Every participant is essential and becomes a creator in creating the auditory and visual environment.
This project is similar to our project in how participants are involved and the core ideas of ‘Group Creation’ and ‘Co-creation’. Our project utilizes sensors for real-time interactions. We offer opportunities for participants to craft their soundscapes freely by interacting with the different sensors and the main controller. The more people interact with the sensors. The soundscape would be chaotic and scary. In addition, participants do not just collaborate with each other in the presentation. They also co-create the soundscape with us by randomly switching between different sound effects, chords, and filters.
Promo Video
-made by Ruojing Chen & Yuan Mei
References
Harlin, R. (1999) The Deep Blue Sea. Warner Bros.
Kentis, C. (2003) Open Water. Lions Gate Films.
Mathivanan, K. et al. (2017) ‘A Study of Virtual Reality’, International Journal of Trend in Research and Development, 4(3), pp. 374-377.
Papersneaker (2019) ‘PLAYLOOP – Interactive Sound & Light Installation’. Youtube. Available at: https://www.youtube.com/watch?v=qvQCBNEG-Rc (Assessed: 15 April 2024).
Spielberg, S. (1975) Jaws. Universal Pictures.
Wong, N. (2023) PLAYLOOP – Interactive Sound & Light Installation. Available at: https://www.papersneaker.com/single-post/2019/09/09/playloop-interactive-sound-light-installation#:~:text=%E2%80%9CPlayLoop%E2%80%9D%20is%20an%20interactive%20sound,speakers%20within%20twelve%20cylindrical%20structures (Assessed: 15 April 2024).
Week11_Final_Presentation_Yuan_Mei
This week, before our final presentation on Wednesday, our project had to make a significant decision. We decided that having a non-linear loop structure would be most advantageous for our installation. This decision is made because the pace of audience interaction with the level button cannot be predicted. We don’t want audiences to rush through all the levels. Such a result would not align with our project idea and goals.
Choosing a non-linear looping structure reduced the user’s risk of moving quickly through each level and prevented the viewer from prematurely ending the immersive experience. Therefore, our project did not end at the last peaceful level, followed by a cycle of four levels starting with the viewer’s interaction with the main controller buttons. This change ensures that audience engagement remains dynamic and allows for a more immersive and controlled experience.
In addition, we received valuable feedback from the audience during the final presentation. Based on the feedback, we took action quickly to enhance our project’s clarity. One insightful suggestion was emphasising the theme, ensuring audiences catch the project’s idea. We promptly designed and displayed a poster at the entry point to address this. This placement ensures that every visitor to the room is immediately introduced to our project’s theme, fostering better understanding and engagement right from the start.
Another valuable feedback suggested relocating the instructions for interacting with the sensors to the top of the box rather than placing them aside. This adjustment significantly enhanced the audience’s experience, making the instructions more readily accessible and easily read while engaging with the sensors. By implementing this improvement, we ensured that participants could quickly understand how to interact with the sensors, facilitating a smoother and more engaging experience for everyone involved.
Additionally, we received suggestions for further enhancement during submission 2. One suggestion addressed the narrow frequency range, particularly the underutilization of the high frequency across some levels. This feedback prompted us to make adjustments to ensure a better listening experience. By implementing sounds in different frequency ranges across the levels, the project will be optimized, having a richer and more dynamic auditory experience for our audience.
Below are some short videos that we recorded during the presentation.
Interaction:
Location setup:
Week9_Testing_Project_Yuan_Mei
This week, our group had a rehearsal to test the audio-visual installation. Because the music store does not have 8 speakers, we made a decision to change the setting from 8 speakers to 4 speakers, building up a surround sound environment. The equipment list was as follows:
– 1 Temprecture&Humidity sensor
– 2 Light Sensors
– 2 Ultrasonic Sensors
– 1 Portable Projection Screen
– 1 HD NEC V302H Projector
– 1 Fireface Audio Interface
– 4 Genelec 8030/8040
– 1 MIDI Controller
– 2 Patch Boards
– 2 Adaptors
Our project will happen in Atrium, and the lights need to be turned off, providing a dark environment for immersion. The sensors will be put on a table at the front. Instructions on interacting with the sensors will be put next to the sensors.
This rehearsal was valuable to us. Because through this testing, we discovered some new problems that we need to fix. For example,
1. The transition between sound effects was not smooth. If participants interacted with the sensor while the sound effect had not finished, the sound effect would directly jump to another one. There was no crossfade, and the previous sound effects did not play to the end.
2. The portable projection screen was too small, so we might consider using a bigger screen or a whiteboard to replace it.
3. We also noticed that if we had four buttons on the MIDI keyboard for participants to enter the four levels, our project would have a non-linear storyline. To make our story linear, we must create one main button for the level control.
4. We also had a problem: the two back speakers did not output sound effects and chords. They only output ambient tracks.
5. The sound effects for the calm level were not output from the speakers for some reason. Only chords happened at this level.
6. The interaction with the ultrasonic sensor is not smooth.
7. The panner has not been used yet. Do we still need the panner?
8. The filter and the master level controller have not been created in Max yet.
We will arrange rehearsals this Thursday, Friday, and next week. Before that, we must fix these issues and ensure a smooth and successful installation process.
Week8_Sound_Creation3_Yuan_Mei
This week, I focused more on creating sound effects for the final peaceful level. I created some relaxing sound effects, including sounds like wind, chimes, water, ice, etc.. The ice cube, page-turning, and rice-dropping sounds are recorded in the studio using Neumann km 184 and the XY stereo recording technique. The necklace sound is recorded in my home studio using Rode NT1 and Focusrite audio interface in mono. The twinkling, water-dropping, and windchime sounds are manipulated using Mconvolution, integrating sounds together to create a new sound effect. All these sounds focus on the high frequency, whereas the sounds for the previous level focus on the low-mid frequency. These soft, high-frequency sounds help build a sense of chill, relaxation and comfort.
Week7_Project_List_Yuan_Mei
This week, I provided a list for our final presentation and thought about what we haven’t done.
Location: Atrium
Setting: 8 speakers (4 on top and 4 on the ground); 1 projector; 1 projection screen
Sensors: 2 light sensors (control colours and patterns); 2 ultrasonic sensors (control sound effects and chords); 1 sound sensor (controls the eq).
Level controller: MIDI keyboard
Arduino: connect to sensors; connect with Max
Max Patch: Jitter (visual); Sound (play random sounds and send the sounds to the speakers)
What we haven’t done:
- Check the equipment availability.
- Check the location availability.
- Sound effects for the final peaceful layer. Add more sound effects if necessary.
- Trim chords and the composition for the rest of the layers.
- Solving the issue of data running.
- Play random sounds in Max.
- Send the sounds to the speakers through Max.
- Set up a main controller to enter different levels.
- Design a head-like shape object to place sensors on it.
Week6_Sound_Design_Creation2_Yuan_Mei
This week, I focused on crafting an immersive soundscape by designing unknown sea creatures’ sounds and hollow sound effects. Employing Mconvolution, I generated animal-like low-frequency sounds for the sea creatures. Some other sounds are from existing sound libruary, which can be used for non-commercial purpose.
I also manipulated violin timbres to emulate the ethereal singing of Sirens. Large reverbs are applied to sea creatures’ voices, delivering an authentic experience of being in a deep-sea environment.
In addition, I incorporated ambient sounds with reverb to heighten the sense of mystery and fear in the third level, emphasizing hollow spaces’ eerie emptiness and darkness. Experimentation with spatial effects further enhanced the immersion, creating a vivid and evocative soundscape that captures the imagination and transports the listener to the fear of the unknown.
Week5_Sound_Design_Creation_Yuan_Mei
This week, I mainly focused on creating chaotic sound effects for the second level of Thalassophobia. The inspiration for designing the chaotic sounds came from my research on the physical influence of experiencing Thalassophobia. The symptoms include dizziness, increased heart rate, out of breath, etc. According to these symptoms, I used high-frequency buzzing and swoosh sound effects to represent dizziness, comprising the chaotic sound design.
I also used low-frequency sounds to represent difficulty breathing, emphasizing a hard-to-breathe deep-sea environment.
Heartbeat with reverb helps to demonstrate the heavy, increased and unbalanced heart rate in the final mix.
To collaborate with ambient music, sound effects are mainly short audio, rather than continuous ambient sounds. Below is the file of the improvised music and sound effects, presenting how the soundtrack might sound through sound installation and participant interaction in our project.
