Virtual Immersion and VR

Environment design for immersion
The designed goal what achieved in this project with a consistent environment by models. For example, the leaves of the trees with pictorial colour shades and wind swing effects. These effects bring impacts the immersion of this project.

Story and time for immersion
The designed story and time decay change leave a participation experience for players to join the story with virtual space and time immersion, guiding players to understand the virtual role and be involved more in the story character.

Visual effects for immersion
The entire story offered multiple styles for players to experience the story. Examples are the accomplishment of animal animations, particles of special effects, and ending nutrition animation, which brought various types for players to comprehend the immersion of this project.

Sound achievements for immersion
The project provides high-quality sound effects to players that players enable them to listen and sense more realistically from a voice perspective. The sound effects assist the entire effects with more accomplishments.

VR and immersion
Waller et al. (2007) discussed cognition and substantial immersive virtual environment (HIVE) with virtual environment technology head-mounted display (HMD). They mentioned that virtual environments (VE) could provide conditions: the three dimensions and first-person perspective. Also, they researched the devices to wear the HMD to analyse the body movement information and orientation in a virtual environment. Waller et al. (2007) also discussed the concept of immersion with HIVE from the device’s display resolution, overlapping rate, headphone, and lightweight aspects.

In this group project, a VR device is to be used as a primary way to express the concept of presence. The project mainly used Oculus Rift, a total resolution of 2160 x 1200 with a 90Hz refresh rate, and it is wearable with a weight of around 470g(Willings,2016). It does not include the self-render system, which could use a laptop to render. However, this function provides a remarkable render condition for this project because the high plant render resolution requirement caused the computer lagging problem.

This group project has not achieved providing the movement function for players, which the VR’s tracking function has not been used. But the first-person render perspective with a three-dimension environment is also built to assist the immersion effects and implementation.

Presence and absence

This project was designed to reach the aim of the concept of presence, which could provide the presence of a digital forest as a tree and participate in the tree’s virtual growing stages. Kundalakesi et al. (2017) discussed the concept of presence is about two concepts which are relative to virtual reality technology “sight” and “sound”, and they mentioned the idea of “being” the immersive environment for the concept of presence. In this project, the presence represents exploring the digital forest environment with the VR technology devices by both visual information and sound and participating in the story, which also provides immersive participation for the virtual world to guide players in the virtual “character”. As the opposite concept of the present, this project designed the virtual world that aims to be a tree which could not be experienced in the real world. Also, participating in the project is the experience that players to be a plant rather than humans. Thinking from a tree’s perspective is designed as the “absence” from the real world.

Apr 27, 2023

1. When making music, I decided to follow the storyline. When the tree is growing and at the experiencing stage, the music is soft and light. This style of music is designed to make players feel relaxed, because the scenes are also relatively healing.

2. The second part of the music is for the fire part. When the tree is burns and gradually dies, the background music is a piece of tragic and sad melody. After that we will add some synthesizer sound effects according to the fire scene.

3. Music of the big tree transferring energy

(1)  The third part of music is used for the reborn part. Based on the principle of the “Wood Wide Web”. When a tree is dying, it passes its nutrients to other trees through its underground root system. So basically their cores are  same. So the third music has the similar melody but with another instrument.

(2)  Haunting Melody: The growth of plants in the forest creates many different sounds, such as the friction of tree trunks and the trembling of branches and leaves. These sounds form a haunting melody, as if telling the story of the forest’s vitality.

Layers of Sound: In the dark forest, the layers of sound are very complex. Some sounds are clear and distinct, such as the chirping of birds. Here, some dreamy bell sounds are used to convey this. Others are indistinct, like the sound of plants growing. This complex layering of sound makes the listener feel as if they are in an abstract dream.

Exotic Flavor: Sometimes, the sounds of growth in the dark forest have an exotic flavor. Here, the environment of the African savannah is used, mixing African drums and ancient chanting to give a mysterious and sacred feeling.

Implying Atmosphere: In the sounds of growth in the dark forest, there may also be some suggestive atmosphere, like predicting the arrival of some mysterious power. For example, our tree network, where tree roots transmit energy, makes people feel a sense of inexplicable tension and excitement.

4. Ending music

The ending music is composed by piano and cello, also is a soft, relaxing major melody.

This music begins with an isolated large tree in a desolate and uninhabited area, only stirred by a gentle breeze. As the melody gradually intensifies, the tree begins to reveal its vibrant vitality, and its branches and leaves start to bud and grow, transmitting the energy of life to the surrounding small trees and grasses.

These surrounding plants begin to awaken gradually, and their weak life force expands gradually, gradually forming a forest. The melody also gradually becomes stronger and more powerful as the energy is transmitted and the surrounding plants revive, carrying with it hope and vitality.

In such music, a lot of vivid sounds can be used to depict such a scene. Here, the sound of piano and cello is used, and as the melody strengthens, these sounds become more complex and rich, evoking the cycles of life, just like the endless evolution and evolution of life.

5. Background music

Natural: The forest’s music typically comprises natural and organic sounds, such as the chirping of birds, the rustling of leaves, and the trickling of water, among other natural elements. It produces a calming and soothing sound that brings peace to one’s mind.

Rhythmic: Depending on the location of the forest, there might be rhythmic sounds of animals or insects like crickets or frogs that establish a steady rhythm. This can give the impression of the forest having its own pulse or heartbeat.

Ethereal: The music in the forest can have an ethereal quality, occasionally, with distant sounds reverberating through the trees like birds singing afar or a waterfall cascading down a rocky outcrop. This creates a mystical and dreamy atmosphere.

Melodic: The forest also acts as a source of melodic music, with birdsong and wind chimes producing a pleasing harmony. The sounds of diverse creatures coming together can generate a unique symphony where every species adds its distinct melody to the blend.

Ambient: In general, the music in a forest can be categorized as ambient since it creates an all-encompassing, natural soundscape that envelops the listener. It can be either calming or refreshing, depending on the individual’s disposition and the time of day.

 

 

 

Apr 23, 2023

Work progress of the visual part:

Have achieved:

1. Scene models and texture(including all plants and other objects), lighting of scenes- Xinyue Lin
2. The seed grows from soil to a large tree(animation)-Xiyue Huang
3. Animals with animations: 5 animals with animation ( including work progress from modelling, skeleton, and animation)-Jiayao Wang
4. Special effects: Rain, fire, snow, glow-worms, dandelions, magic fire-Amber Zhang
5. day and night transition(skybox)- Xiyue Huang
6. Nutrition transition animation(roots extending animation and nutrition transition with blooming effects)-Qiandai Sun

Have not been fully achieved:

Unity:

1.Unity file combination: Combine the sound and all the above visual parts, and test all the parts work successfully.

2.Unity interaction: to trigger when the player’s vision moves to the area: including the 2.1animation, 2.2special effects, 2.3 day and night shift, 2.4 fire and transfer the next scene(to trigger the nutrition animation) (-Amber Zhang, glad to if anyone participate as well)

Oculus Rift:

1. Might expect to test the Oculus with controllers to check if all devices could use the entire scene with the play mode fully successfully.
2. Might expect to set up and test the controllers’ functions and buttons for the interaction in Unity.

Apr 2, 2023

To match the style of our game, we decided the sound in this game  will basically consist of real ambient sound and some synthesized sound effects and music.

For the recording part:

We have recorded some ambience sound in outdoors. Birds, insects, water.

 

Foley part:

Materials:

Dead leaves, dead flowers, ivy, soil, pipe, rice, tissue, crushed chili, tin foil, celery, bubble wrap, cornflakes, pill tin packaging, effervescent tablets, paper and paper box, electric toothbrush.

We used two kinds of microphones to compare the different recording effects, one is a dynamic microphone and the other is a condenser microphone.

 

Plant growing sound: for the tree growing from soil to ground sound, we used some fibers and twisting them. The sound of snapping celery is used as the seed crackling and drilling out of the soil.

Leave sound: We used some sounds of rubbing plastic as the sound effect of leaves shaking.

Fire sound: To simulate the big fire, I use a combination of the sound of rubbing tin foil, crushing crisps, twisting bubble wrap, breaking off celery and small size impact and whistling wind sound to create more layers of a fire. By adjusting their frequencies and proportion to achieve a balance.

 

Music part:

When making music, I decided to follow the storyline. When the tree is growing and at the experiencing stage, the music is soft and light. This style of music is designed to make players feel relaxed, because the scenes are also relatively healing. The second part of the music is for the fire part. When the tree is burns and gradually dies, the background music is a piece of tragic and sad melody. After that we will add some synthesizer sound effects according to the fire scene. The third part of music is used for the reborn part. Based on the principle of the “Wood Wide Web”. When a tree is dying, it passes its nutrients to other trees through its underground root system. So basically their cores are  same. So the third music has the similar melody but with another instrument. The ending music is composed by piano and cello, also is a soft, relaxing major melody.

 

Code Part:

Many sound events follow the events in the unity scene, such as the sound of burning fire, raining , so most of the functions of playing sound are through “AkSoundEngine. PostEvent(“Event name”, gameObject) in C# inside the unity project.

RTPC in C#:

Because the change of our unity scene is not achieved by the timeline code, when using RTPC to change the ambient sound day and night, I cannot directly correspond the  RTPC to time, so with the help of our tutor, we use “0” and “1 ” to correspond to “day” and “night” events.

VRController button sound：

We want to play the sound of the tree growing when the VR controller button is pressed, because pressing the button is the player’s own control of the growing process.

The code we used(But not worked, still not to debug):

void Update()
{
if (Input.GetButtonDown(“ButtonName”))
{
AkSoundEngine.PostEvent(“Play_Growth”, gameObject);
}
}

void Update()
{
if (Input.GetButtonUp(“ButtonName”))
{
AkSoundEngine.PostEvent(“Stop_Growth”, gameObject);
}
}

Mar 18, 2023

In the end we discussed and decided on the final plot. The player, as a tree, and experiences the life of a tree from the tree’s point of view. From sprouting to death, and eventually feeding other trees with nutrients through the Wood Wide Web. This in a way means rebirth. And as a tree, the player has more visibility and more interaction.

This is the storyboard link:

https://docs.google.com/document/d/1NL8X9pe7b3pm9L8X6YRv2nq0pACQ6yUOT4ZK-qC92y8/edit?usp=sharing

I m not sure sound part, please sound design classmates can help me finish the sound parts.

Mar 15, 2023

Based on the feedback, I changed some of the episodes to better convey our central theme. Here is the storyboard I drew up.

 

Mar 15, 2023

Qiandai Sun：

My inspiration comes from the food chain in the ecosystem. Players experience the perspective of flowers from the beginning, sprouting from the soil, growing out of the ground, and being eaten by rabbits. Then start to experience the rabbit’s perspective, explore on the vast grassland, you may meet its family and friends, interact with them, and then be eaten by flying eagles. Then the perspective turns to the eagle, which catches rabbits to feed its children every day, until one day it is too old to fly and dies on a cliff. Its corpse was decomposed by microorganisms and entered the soil, and the perspective returned to a seed in the soil.

Mar 1, 2023

The whole experience is from the first view of the flower. The viewpoint has the life value of the flower. The sound could be like Hoopoe in Catching Fire, showing the emotions of the flower.

The flower grows from the earth and is later picked and taken home to the children of the family. The flower is planted in a pot by the child and is observed in the third person perspective of the family. The child studies, his mum and dad go out to work every day and his grandparents come to their house at weekends. (Here you can then watch childhood, middle age and old age.) Ways to increase interaction: for example the flower can wilt if it wants attention 🥀 so that the owner will come and water it and take it out into the sunshine. The owner can exercise in the sun while the flower basks next to him and watches him, and tries to do the same actions as the owner. Watch people’s lives from the flower’s first view and see their own life value. As time passes, the flower withers and the old man, for example, dies. The flower is eventually placed in the old man’s coffin, but as it is being placed, a wind blows and a petal floats up. The view follows the petal as it drifts higher up, overlooking the funeral, and finally drifts into the distance. The image fades and then reappears in the earth.

 

想法：

全程以花的第一视角体验。视角中有花的生命值。声音可以像《捉妖记》里面的胡巴，表现花朵的情绪。

花从土里长大，之后被人采摘回家，送给家里的孩子。花被孩子种在花盆里，以旁观视角观察着一家三口的生活。孩子学习、他的爸妈每天出门工作、爷爷奶奶会在周末来他们家。（这里就可以观看童年、中年、老年生活）增加交互的方法：比如花朵想让人关注就可以变蔫🥀，这样主人会来浇水，并且带它去阳光下。主人可以在阳光下运动，花朵在旁边晒着太阳并看着他，并且努力和主人一样做动作。以花朵的视角观看人们的生活，并且能看到自己的生命值。随着时间流逝，花朵逐渐枯萎，老人比如也去世了。花最后被放在老人的棺材里，但是在放的时候，有风吹过，一朵花瓣飘起来。（花朵的第一视角看到自己飘起来，但是花的枝干还在坟墓里）视角跟随花瓣向高处飘，俯瞰葬礼，最后飘向远处。画面渐隐，黑屏。然后又出现在土里。

Feb 25, 2023

Amber Zhang:

Keywords: Presence, life cycle, Nature

Concept: user experiences options from different natural creatures.

My idea: the initial scene would start from in the abstract universe but correlate with natural spaces. The start point might launch from the abstract universe scene that a soul shape human would appear. The player would pass the abstract universe scene where consist by abstract trees, grass, mountains etc. There would contain three doors that each door would lead player to the next different scene to nature(Our group discussed the idea to open the door from a dark corridor to enter to the next scene when the first and second meeting. The initial open-door idea to go to next environment for our group project I remember should came from the Xinyue Lin or Qiandai Sun) The first door would be the scene that player would as the flower from a seed to bloom and might until nature decay. The second door the player might as a human that might experience the infant, young, old three age span. The third door the player might be a bee, would fly over the earth’s ground. The person’s life span and flower could still prolong the previous our group project finalised idea, and for the human stage the detailed part, I might prefer my previous suggested idea on discussion board1 from the blog(From Amber Zhang part on discussion board1: “My brainstorming suggestions for the three stages of life span: my brainstorming idea would base on the group discussion result yesterday(Wednesday): the group project would only focus on one of the life span about the young age to explore, and pick up objects only for rest of ages(infant and old) , I think for the young age could represent via one scenario: the main character could display in lecture space: the scene exploration design could be: the sunlight could be warm outside and flash light shining from window, the leaves could drop from the tree, it could be the Autumn season. The main character would start the view from setting on the chair and books and notes on the table (the view angle displays the book pages on table). The main character enables to stand up and walk around the lecture room. There could be notes on whiteboard, but no other people would be in the lectures. The lecture might be a biology class; the whiteboard could display sketches drawing of flowers with key points and learning knowledge, and one reminder on whiteboard could display: a test on Thursday. The main character enables to walk towards whiteboard to check the contents of knowledge on whiteboard and explore the entire lecture room. The lecture room could be big with basic devices: computer, projector, big window, desks and chairs, whiteboard. The background voice: the main character could hear other people’s talking from outside of lecture room, the ring bell sound, the background noise sound.When the time duration approaches to time limits (for instance: 3 mins), the scene transection trigger would display. And the voice might appear as well to tell main character the guidance to move to next scene.”) . The bee’s scene might end from flying motion. After each scene ends, the player would go back to the initial scene: the abstract universe to alter back to the soul to choose again. However, I might suggest each player merely have 15 mins to explore the entire story instead to loop the option to experience tedious.

After the soul transferring the different role from different nature species, the three different life spans could be considered to represent how the presence is performed. Meanwhile, the three format creatures are the three diverse ways to demonstrate the life cycle.

Feb 25, 2023