Visual Style and Sound Design: Part 1 – Philosophy (s1850658)

At the time of writing, there is still a decision to be made on what the visual style of the project will be. When discussing this in the previous meeting (Thursday 2nd February), it got me thinking. Different visual styles would necessitate very different things in terms of the soundscape and sound effects.

The main two styles that came up in this meeting were LowPoly and something more realistic. Upon hearing this, it made me want to create some prototypes/mock-ups of what the world could sound when presented visually in these two styles. When considering what to specifically try and represent to get my ideas across, I settled on the flower: the soundscape surrounding it and the sound effects that accompany its growth. This setting would require sound design that is quite atmospheric, by virtue of the fact that the player is unable to move.

To represent the Low-Poly style, I felt as though the soundscape shouldn’t be a realistic representation of what these things would sound like in the real world. Instead, the soundscape and sound effects should aim to represent the visuals in a more indirect manner. I felt as though the soundscape should be an underlying drone, like a pleasant chord. The wind could be represented by some high notes, synthesized or otherwise (or perhaps even created using white noise), that fill the highs (that the sound of a realistic breeze would usually occupy). These sounds could whoosh in and out in response to the flower and its surroundings moving with the wind. This opened the possibility to have the stages of plant growth be represented by fluttering notes playing an arpeggio that fits with the chord of the underlying drone that accompanies the scene. I believe that the use of a low-poly visual style requires sound design that aims not to faithfully re-create sounds from the real world, but instead create something that represents the visuals in a more abstract manner.

When working with a more realistic visual style however, I felt as though an almost opposite approach was appropriate. The soundscape of the scene should reflect what a person would hear if they were standing in the scene as if it were a real place. Every sound in the scene would need to approximate its real-world counterpart. I say “approximate” because for example, plants don’t make any sound (audibly at least) when growing. Any sound effects for the plant growing would instead have to be a semi-realistic representation of what we would expect it to sound like if it did exist. When choosing a realistic style instead of low poly, many aspects of what we are trying to get across through the experience change. When working in low poly, I feel as though the aim is not necessarily to fully immerse the player in a realistic world, whereas in a more realistic visual style I feel as though the aim is to try and fully immerse the player in a believable reality close to our own. Low poly allows for more flexibility when it comes to representing what’s on screen. A realistic visual style however, demands much greater attention to detail in order to create a convincing world. Realistic requires emulation of the real world; low poly allows for the creation of a new one.

A question this discussion raises is “once a visual style is chosen, what happens to the audio work for the other”. I would argue that there could perhaps be a mix of these two sound design philosophies in the finished product. This could be because the visual style ends up being some blend of the two (such as a cel-shaded style), or perhaps more abstract sounds could be used to represent things that don’t have a real-world equivalent, even if the visual style is aiming to be more realistic.

Audio Player

lowpoly

The First Meeting:

Xinyue Lin:

The concept of this project is to turn the viewer into a plant, such as a tree or a flower. Participants will have the opportunity to see and feel life from a plant’s perspective, rather than a human’s. For example, they can experience wind, rain and sunshine, watch bees gathering nectar, birds flying overhead, insects foraging and more. Viewers will also witness the growth stages and life cycle of the plants, such as the elongation, flourishing and withering of leaves. The project will also incorporate sound effects, for example, when an insect flies onto a flower, you can hear the sound of the insect as clearly as pedaling. You can hear the sound getting further and further away as the insect flies away from the flower.

The Second Meeting:

Amber Zhang:

Feedbacks from Lecture about group project(might not fully covered all feedback points)

Inspiration suggestions are from Dr. Jules Rawlinson: Presence vs absence? (How might the project achieve both concepts?)

Inspiration research area suggestions are from Dr. Jules Rawlinson: academic research key words: “death and digital presence”, “VR grief representation resolution”

Inspiration example is from Dr. Jules Rawlinson: looking back (Notes: the project’s name was not fully sure is correct)

Inspiration suggestions are from Mr. Leo Butt: “Death” could be repeatable in digital games; players could be reborn and learn from “death” in digital games?

Target audiences: will our project still aim at treatment of mental issue such as stress, grief?

Perspective idea of soul transition in our project? becoming a soul of Plants and humans?

Immersive technologies installation agenda? AR VR Projectors/real simulated physical effects: rain, sunlight?

The Third Meeting:

Xiyue Huang:

idea about ‘Death’&’Presence’

What is presence, I thought of death, the antonym of presence, what is existence. The idea comes from coco. Although people are dead, they have existed before. If their memories are remembered by others, they are not really dead. If they are remembered by everyone, this is presence. Another conceptual inspiration comes from COCO. This is an animated movie about death. The real passing is not death, but being forgotten.
Purpose: To achieve a spiritual healing effect and heal the soul. Death is a part of life, people always ignore its existence and can’t believe it. But one thing is clear, we all have to face it and experience it.

I want to set up a memory palace scene, the overall space is clean and simple, and there is a part of the VR space in front of the player. For the time being, there are currently four rooms, and each room represents the memory of a dead person. Through the interactive experience with this room, players can feel what kind of person the owner of this room is. Let’s use sound to express the human state, visualize the sound and make it into a waveform.

Model style / Keywords: fantasy, simplicity, color, line

<soul>

After the third meeting, the professor questioned the idea of the theme of death. Compared with the theme of flowers, this idea seems to be not enough for users to participate in immersion. The antonym of presence is absence. It doesn’t make the player feel the presence, the player’s experience is absent, and we finally settled on another theme.

Low Poly and Sounds

The wind sounds heard in the low-poly example were created in a Max MSP patch, using a series of noise~ objects going through resonant bandpass filters. Over time, the cutoff frequency of each noise oscillator’s filter would gradually shift, along with the resonance, and the volume. The stereo split between left and right for each would also shift over time. I let the patch run and recorded around two minutes of it, then picked a section that I liked and used that in the example.

The soundscape in the background was made using a combination of a Serum patch and a free box violin VST for kontakt. The Serum patch arpeggiated maj7 chords, while the box violin sustained the notes. Both were fed into Michael Norris’ spectral averaging plugin, with a little reverb added as well.

For the ascending sound that indicates that the plant has grown, I again used a Kontakt instrument – this time a music box. I filtered out some of the highs, added some tape warble using SketchCassette and then some reverb.

This is a storyboard about how to combine two ideas (plant and death). I add the person’s perspective after the flower, and I put an animation to make the perspective change more smoothly. People pick the flowers to bring to the funeral. The vibrant flower actually died when it was plucked, but when it was put into the coffin, the perspective changed to that of the person lying in the coffin, experiencing a person’s life from the perspective of a person. When life comes to an end, when death is ushered in, and there is darkness in front of my eyes, I hear the sound of rain again, and then I find myself as a flower seed in the soil. This story is about a life loop. The Bible mentions ashes to ashes, and dust to dust. This sentence gives people hope to restored to eternal life. From a biological point of view, A series of biochemical events occur after death, the end result is a kind of “biological reincarnation” into millions of bacteria, insects, plants, and other living organisms. It is a kind of “rebirth”.

Ref:

Howard, G. C. (2021) ‘Ashes to Ashes and Dust to Dust’, in The Biology of Death. [Online]. New York: Oxford University Press. p.

Archer, G. (2013) Oxford Dictionary of Reference and Allusion. Refer 29 (1) p.part–par9.