Concept

The combination of visuals and sound in films brings us an immersive experience, where sound plays a crucial role in shaping immersive spaces.Michel Chion categorizes film sound into two parts in “Audio-Vision: Sound on Screen”: Diegetic, which is sound within the narrative space, and Non-Diegetic, which is sound outside the narrative space. The interplay and transformation between these two types of sound create a sensory immersion for the audience.Meanwhile, Ben Winters mentions in “The Non-diegetic Fallacy: Film, Music, and Narrative Space” that Non-Diegetic sound is partly a sign of the fictional state of the world created on screen.Therefore, can we apply the theories of Michel Chion and Ben Winters to art installations, making sound an indispensable part of sensory immersion and allowing sound to work with visuals to create a field within the “Place” of this installation?

Idea

Sound is divided into two parts: Diegetic and Non-Diegetic. Diegetic refers to Place Sonification, while Non-Diegetic refers to LiDAR Sonification.In Diegetic sound, we use sounds that are closer to the real world, which can give the audience a sense of familiar unfamiliarity. This type of sound can shorten the distance between the audience and the space of the installation.In Non-Diegetic sound, we primarily use sounds from synthesizers, which are very distinctive and can quickly capture the audience’s attention.Through the combination of Diegetic and Non-Diegetic sounds, the entire installation’s field is placed at the intersection of reality and virtuality, making it both real and beyond reality.
For our simulated Edinburgh immersive experience, it’s essential to employ strategic spatial audio techniques. To achieve this, our recording plan incorporates the Ambisonic format for mixing, with the aim of utilizing a 5.1 (or 5.0) surround sound playback format to immerse the audience.

2. Field recording

Recording Plan

Dual Microphone Sets

• Stereo(ORTF): This set will employ the ORTF stereo technique to ensure high-quality 2-channel stereo recordings. This approach is aimed at capturing a rich, natural sound field.
• Ambisonic: In parallel, we will use an Ambisonic microphone to capture sounds in the first-order Ambisonic format. This technique is chosen for its ability to preserve the spatial characteristics of the soundscape, offering a more immersive listening experience.

Recording Perspectives

• Distant Shots: Aim to capture a broad spatial layering, retaining the scene’s depth.
•  Medium Shots: These should align with everyday auditory habits, ensuring there are plenty of dynamic changes.
• Close-up Shots: Focused on detailing specific sounds within the environment, these recordings will be more targeted, serving potentially as sound effects or sonification elements, rather than general ambience.

Impulse responses

• To record the acoustic reverberation of different places, we plan to use a slapstick and balloons to create impulse responses, allowing us to capture the unique acoustic signatures of various spaces effectively.

Equipment List

1. Microphone
Sennheiser AMBEO VR mic
https://bookit.eca.ed.ac.uk/av/wizard/resourcedetail.aspx?id=4149

Schoeps – MK4 + CMC1L * 2

4. LiDAR Sonification – Nondiegetic

LiDAR, as a contemporary high-tech innovation, has inspired audiovisual installations that often employ nearly sci-fi sonic textures to reflect their technological essence. These installations predominantly use synthesizers and processors to create sounds, aiming for an auditory experience that matches the visual in its capacity to transport audiences beyond the familiar. The sound design in these setups focuses on dynamic and timbral shifts that keep pace with visual transformations, emphasizing rhythm and atmosphere rather than conventional melody.

https://vimeo.com/649452074

An notable example in this domain is Ryoichi Kurokawa’s “s.asmbli [ wall ],” where sound plays a pivotal role in creating the atmosphere and making the LiDAR scanning process audible. The installation capitalizes on the rhythmic aspects of the scanning to set its tempo, relying mainly on synthesized sounds without clear melodic content. It employs reverberation automation to reflect changes and progressions within the scene, enabling the audience to intuitively understand the creation and evolution of data. This immersive experience guides viewers through the construction, transformation, and dismantling of various places, driven by the combined narrative of sound and visual.
The adoption of technologically derived timbres is intended to signal the processes underlying the visual display, employing nondiegetic sounds to hint at the artificial crafting of these places. This strategy not only deepens the immersive effect by linking technology with creativity but also prompts the audience to consider the interplay of human and technological efforts in depicting and altering environments. Installations like Kurokawa’s encourage exploration of the relationship between organic and technology, showcasing the artistic possibilities offered by LiDAR.