After some weeks of discussion and have decided on the project’s foundational concepts, understanding and structuring its sonic possibilities and defining the respective tasks seems appropriate. To quickly sum up the foundations of this project, the project will be composed of an exhibition, where an immersive experience of a pre-captured space (with LiDAR) will be displaced that users will explore through interactive methods. Through this brief, and after careful thought, the collective can identify four main sound-work domains:
-
-
- Soundscape Capturing
- Place Sonification
- Interactive Sound
- Installation.
-
This is handy since the group comprises 4 Sound Design students, to whom these sound-work domains can be later assigned as roles.
Soundscape Capturing
The relationship between the concept of place and soundscape is crucial and interdependent. A place can be defined as a location with unique physical, cultural, and social characteristics. On the other hand, the soundscape refers to the acoustic environment that a person experiences in a particular place, including natural and human-made sounds. The soundscape contributes to our perception of a place, creating a sense of identity and atmosphere.
Capturing and reproducing the soundscape along with the place (through LiDAR) will play a critical role in shaping perception and understanding of the presented place. Therefore, it will probably be one of the first performed sound tasks for this project. The defined location of capture is sonically vibrant, having an urban profile sided with nature. It is composed of an extensive collection of elements that must be carefully thought out across all capture moments: planning, recording, and editing. This being said, through professional field-recording technology and carefully thought-out techniques, we propose to explore ways of best capturing this soundscape, having in mind its high-fidelity representation and the means of the entirety of the project along with its final form.
Place Sonification
One of the project’s core concepts to capture a location into point cloud data urges a new sound opportunity. A critical rule taught through the course and that here applies greatly: where there is audio, there is data, and where there is data, there can always be audio.
One aspect of sound art is data sonification, which involves transforming data into sound to make it more accessible and to enhance its representation. In data sonification, data points are mapped to specific sound parameters, such as pitch, frequency, and volume. This creates a sonic representation of the data that can reveal patterns, trends, and relationships in ways that might not be immediately apparent through visual means. Sound art that employs data sonification can range from immersive installations to interactive pieces and has the potential to offer new perspectives on data and provide a unique and engaging form of artistic expression.
This being said, point cloud data can be turned into quantifiable formats such as XYZ and RGB and exported into an xml. file. The resultant xml. file can be converted into txt. file and be fed to a Max/MSP patch that reads these numbers. From here, through computational sound composing, data will generate a meaningful sound-art piece that represents the recorded place. In a sense, letting the physical shape of the site perform a new soundscape. This method could also play along with the recorded soundscape audio, where this high-fidelity representation can be processed and re-shaped according to the point cloud data sets.
Interactive Sound
As stated previously, the user will perform some degree of interactivity with the exhibition. Users will interact through a control system that takes in a relatively simple gesture (stepping up or down). Although simple, this represents a decision-making process of the user, which on its terms, presents multiple sets of data:
-
- outcome A
- outcome B
- similarity to a past outcome
- dissimilarity to a past outcome
Such parameters can be represented by quantified data and be sonified through Max/MSP. Perhaps one other form of sonification would be to use unity/unreal for this purpose, which along with middleware like Wwise or Fmod, could work sound in exciting and varied ways. This task, however, will demand the crucial point of finding the optimal way to read live data from this simplistic control and manage a system that is optimal for such a purpose.
Installation
As stated previously, the final presented product of this project will be an exhibition, in the sense that it will use space to deliver an audio-visual product. This premise demands the installation factor for both Audio and Visual domains. Now, it is also clearly stated that the project proposes an immersive experience. On the one hand, this “immersive” label narrows down the installation options. On the other hand, it demands innovative and creative solutions.
Installation is the task that, for the most part, is thought out as the last step when it is not. It plays with various factors that must be planned, measured, tested and improved. It accounts for the type of audio to playback, the space being used, and most importantly, the listener, both as an individual and the audience. The installation must be thought out early in the project and improved along the project developments and must account for all the different needs that might occur over time.
Two possible solutions come to mind for the “immersive exhibition” concern of the project: a multichannel surround sound speaker set-up or an ambisonic solution with a binaural format. However, although ambisonics provides powerful ways to create immersive experiences, it also demands more resources and would not offer a simple solution for the different possible orientations of the listener. Therefore, a multichannel surround sound system seems more appropriate as this can more easily be adapted and tuned to the ongoing project’s needs.
