As mentioned in previous posts, the project’s interpretation and exploration of the concept of “place” starts with using point-cloud formats. The performed data export out of this format into text values (read the blog post “Sonification 2# – Interpolation and Data flow”) allowed us to understand the genesis of a point-cloud file and break down its meaning: a collection of points with a set of spatial coordinates XYZ, colour coordinates (RGBA) and an Intensity value (Reflectiveness). This understanding closely reminded the painting technique of pointillism. This technique uses single strokes or dashes of paint, each with characteristics (e.g. position on canvas, colour and texture) to paint the bigger picture.
The idea of a complex texture made of repeating the same gesture, yet each with different characteristics, naturally fell into the sonic granular synthesis technique. The sonic nature of granular synthesis seemed to fit the sculpting method of a point cloud better while also providing an exciting set of sonifiable parameters. It could also use the actual soundscape of the LiDAR recorded place, which seemed meaningful since this meant that the entirety of the sonification would come from recorded data of the defined site (the News Steps).
The granular system bases itself on a Max/MSP patch provided last year by Dr Tom Mudd following the means of the Creative Coding course’s academic tools and resources up for grabs and further developments (Figure 1). This patch provides a way to trigger an audio buffer or a snipped-off while allocating this to a separate voice, collecting a max polyphony of 16 voices. This patch also provides a transposition factor (in semitones) t. Once triggered fast enough using a metronome, a sonic texture is formed. As described by the now, 4 sonifiable factors can are set:
-
-
- Rate of grain playback (metronome rate)
- Starting point within
- Grain length
- Transposition
-
It seemed essential for the starting point of the sample to be defined by the Z axis, for the sole reason of having sorted the data sets from “top-to-bottom” (read previous post “Sonification 2# – Interpolation and Data flow”), which allowed this parameter to still somewhat linearly scroll through the entirety of the audio buffer and be easily synced to the other granular channels, something discussed further down this post. Along with the “grain” starting point comes its length, which was set to oscillate along with a scaled B coordinate parameter between -5000 milliseconds and +5000 milliseconds, having on offset at 7000 milliseconds. The “grain triggering velocity” performed with a simple metronome, whose rate was set to follow a scaled value of the “Point Number” to oscillate between 100 milliseconds and 5000 milliseconds, a fluctuation that both fit well with the 16 voice polyphony and provided a consistent yet somewhat irregular texture. Finally, transposition followed the Intensity value, a decision made upon its very irregular and significant variation, which gives an interesting spectral composition to the polyphony.
The overall project is structured around a substantially more complex layout than a single use of one of these systems. As mentioned previously, an immersive dimension to the exhibition was set to be achieved through spatial recording techniques (2x A-B setup) in combination with a 7.1 surround sound setup. As such, this same patch is repeated over the eight distinct audio channels, where each has its specific audio buffer feeding the granular synth.
In previous planning, it was also set that there would be a user interface to control some moving in/back in time. This idea was defined to be represented through a system that allowed the processing of audio files specific to a timestamp in the day and that could replace the base audio samples with earlier or later ones in the day. This system meant a switch-type transition between the two files, where one replaces another on command, which usually would sound instantaneous and abrupt. To avoid this and instead develop slower morphing movement, for each 7.1 channel, two granular patches are set to be crossfaded between them once triggered. In this crossfading system, a function to clear the respective audio buffer when one’s signal matches zero was put into place, giving space for a new one. This interactive audio replacement system will be in-depth documented in a future Sonic Interactivity post.
The following blog post will review the heavy processing of this granular sound in parallel channels down the signal chain. A delay loop and a reverb set work together with the sonified data set to generate the other-worldly nature of this sonic work.
Many exciting sonification, methods and sounds are to come to this blog, and as such, we will keep you posted.
If you’re reading this in order, please proceed to the next post: ‘Sonification #4 – Sonified Processing’.
David Ivo Galego,
Head of Sonification.
