Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.
Due to the holidays, the school's projector is no longer available for hire. We had to use our only projector to set up the scene and shoot the video. As a result, the video recording did not meet our expectations and we had to project and shoot the video separately and edit it to create our final presentation.
After the first presentation, we optimised the project, rebuilt the scenario and recorded the video again.
The video was shot by Rudan and Jeala, the video was edited by Rudan, the sound and music production was handled by Yuan, and the set was built by all of us.
After collating all the live recorded videos, I drew up a flowchart of the video clip and also the subplot for that video clip.
(This image was drawn by Rudan Zheng)
The first act of the video starts with a ripple and ends with a ripple in the title as well as an introduction to the project.
The second segment is three two-second videos switched with the sound of water drops, used to quickly demonstrate three visual effect clips from our project, similar to the abstract section of the thesis.
The third section is the introduction to the project, and for this clip I used typewriter effects and sounds to create it, which seemed to make the video a little more interesting.
The fourth part is our production process. I’ve sped up the video because it’s been a long process over the last two months or so. The main part is the production process of building the set on the day we exhibited it, then interspersed with a roll of film strips of different types of production processes such as software, testing and material assembly.
For the part about how to transition to the actual display, I tried a more subtle approach with the last video in the rolling strip of film being a video of the characters entering the display room. This video gets larger as the figure gets closer to the display until the moment the door is pushed takes over the whole frame. This allows for a very natural transition to the actual part of the display.
The fifth part is the actual exhibit, and because it is so long, I have only taken one minute and 30 seconds of it for the screening part of this video. Because of the range of the shot, we only got the main view of the wall, and the other two walls were projected using a small screen to show the image in the video.
We have included an interview section, as well as an interview with the audience about the project after they had visited us. The interviews include but are not limited to, their thoughts and what they think we could improve.
At the end of that video, I created a scrolling bar of people, which included the division of labour within our group and what we were working on. The background image for that section is a poster that I created to enrich the visuals.
The content of this blog consists of the work of Rudan, Jin, Yuan and Jaela.
Space
Although we understand the space (E15A) for the presentation is not that big and neat enough for our installation, we still choose the room for our update project. That is because there are several rooms booked for the degree show such as C09, C02, and other rooms such as C08 and E21 are not available during the spring vacation. Therefore, we used a large black curtain to cover the floor.
A possible future improvement would include a more suitable room for the audience to walk around. At the same time, it is necessary to be well-lighttight for projection and proper size for filling the room with music.
We expect to find a room large enough in the future that the projections of water ripples and the interaction of the touch designer can fill the room. Max is still capturing the movement of the characters, but these changes will present a more dramatic visual change. The changes made to the image by the characters will become more visible
installation
The current installations appear too crude and in the future, we can improve the visual effect in terms of the beauty of the installations, the neatness of the environment and the refinement of all aspects.
Interaction
Although we enriched the interactive way, the experiences still be regarded as insipid. In this case, it would be better to make two video walls interactive rather than just videos with masks. For example, the audience can be captured as a human mask.
As well as this, the current interaction is still between the person and the wall, and the person and the installation, we want to make it so that the movement of the person has an effect on the whole room. For example, when people walk on the floor, the floor will ripple as they fall; the water ripples on the walls will change as more people talk in the room, the volume of sound, etc.
We expect the future of interaction to be a dual interaction between people and people and people and things.
Music
We have updated another piece of music, but the main part still be remained. The original piece is generated for the vibration of water, so it tends to have a strong sense of rhythm. It would be better if we change the style and atmosphere from sci-fi music to sound conveying pleasant and peaceful.
Convey core ideas
We have been working on this problem for a long time, but the final effect was still not satisfactory. Apart from the introduction before watching and the explanation from us, the designers, it seems to be difficult for the audiences to understand what we want to convey and the metaphor for each piece of content. In the future project, we would prefer to learn from more projects which can be easily understood.
After the show was finished, we made some more adjustments by analysing the feedback. For the music, I added two melodies, a piano piece and an ambient piece for the video clip background and the interview background music. To make the music more amplified, adjustments were made to MAX,
adding the amplitude of the music to make the connecting water tank ripples more visible and the changes in the water tank ripples more distinct with the rhythm of the music.
Each beat of the music brings about an ebb and flow of the water. Combined with the images we have made, it allows the audience to be immersed in the beautiful music and enjoy the beautiful blend of water and music. As the climax of the music gradually approaches, the characters on the projector run faster and faster, and the amplitude of the water tank ripples increases, creating a beautiful wave of water as if it were an artistic painting. People followed the rhythm of the music and felt the beauty of it all as if they were immersed in a beautiful dream world. In this live experience, the effect of the images is not only created by the vibrations and intensity of the music but also changes according to the sound made by the audience through the microphone radio. Participants can shout, jump, snap their fingers, clap their hands, sing and other interactive ways to make the images change even more. This unique way of interacting with sound creates an overwhelmingly exciting, entertaining, and creative atmosphere. Finally, when the music slowly ended and the water tank ripples slowly calmed down, the whole space returned to peace and calm. This live experience gave them a musical and visual treat and a unique experience and enlightenment in their minds.
Based on our meeting on April 19, we decided to make the installation more interactive and playable. I find some video tutorials of Touch Designer with Kinect and Leap Motion on Youtube. Since the Leap Motion is not available from the university, I booked the Kinect camera from the music store.
From my perspective, the theme or style needs to keep consistent with our project, which means it is expected to be related to water and water-ish stuff. Therefore, I prefer creating water painting effects through the feedback function and the slope TOP.
I use kinect v1 TOP and kinect CHOP of panel and selector to get the left and right hand data from camera. In addition, I drag the data as a chop reference to the position of the water painting circle.
And then I combine this part with the sound visualization. Now the sound visualization part would appear when the part is covered by the circle.
The content of this blog consists of the work of Rudan, Jin, Yuan and Jaela.
According to Philly's and Jules's feedback on our presentation and exhibition at ECA Main Building E15A on April 19, I noted them and some corresponding ideas and held a meeting with my team members.
Feedback Record:
From Jules:
The sound generated by Maxmsp is an active feedback process. The waves can travel across the water’s surface and hit the other end, and then it started to come back again.
The sound is expected to be louder to avoid being broken up. The speaker is not big enough to produce the high frequency and create the vibration. Also, the interaction way of clapping or other ways to make noise is not un-stopping. Try to actively use feedback to make it stable.
Refer to the Chandni pattern.
Create sound or fans to create ripples on the surface, and also make the strings move.
Animate the videos, make the character moving from one screen to the other. The person can walk around, disappear and so on. Merge these three screens.
The music feels like a sci fi music rather than a positive, pleasant atmosphere. Add a bit variety.
From Philly:
Working in a slightly bigger space that was more nevigable, allowing the audiences to walk around. Present this piece as a more walkable and immersive installation.
Place objects or interacive elements within the space.
The video materials did not effectively communicate the themes of the piece. Create significantly more variety and use some more abstract imagery.
The touch designer aspect needs some more variation.
More proposed theme in video content, music evolution or visual evolution.
I summarized the feedback and categorized them into several parts given by Jules and Philly, and organized a meeting on April 19 to discuss the update plan.
I found some references as follow:
After the meeting, we decided to make some improvements as follow:
Yuan will generate more peaceful and pleasant sound pieces through GarageBand and other softwares.
Jin will keep working on the digital ripple with Touch Designer, and generate a new interactive way for the audience, which is a water painter effect followed by gesture interaction based on Kinect camera and feedback function. In addition, Jin is going to prepare the manuscript and voice over for final video.
Rudan prepared the new material, refined the video of the characters walking around, and edited the final video.
Jaela is going to prepare the video editing and video recording.
Reference
Abstract speech visualisation – touchdesigner tutorial 45 (2021) YouTube. YouTube. Available at: https://www.youtube.com/watch?v=1FgJ842dyr4 (Accessed: April 27, 2023).
Touchdesigner Tutorial edge feedback (2021) YouTube. YouTube. Available at: https://www.youtube.com/watch?v=hve2UbKgJ9s (Accessed: April 27, 2023).
The content of this blog consists of the work of Rudan, Jin, Yuan and Jaela.
Preparation
Selection of rooms and booking of equipment:
Jin booked some rooms in ECA and evolution house to test the effect. For example, ECA C02, E15A, E15B, B55, Photography Studio G09 (Evolution House) and so on. Taking into account factors such as shading, the effect of room size on sound, projection and so on, we finally chose E15A as our final exhibition room.
In the meantime, we hired and tested some projectors. We wanted the image to fill the whole wall, so we used a short throw projector for the main image.
Step 1:
Debug the software and change the details. For example, the intensity in Touch Designer is connected to the visual audio cable, the order of the music in Max is adjusted, etc.
Step 2:
Make the water tank, cut the PVC sheets and glue the PVC sheets with a glue gun.
Step 3:
Place the water tank. As we needed to light the tank at a 45 degree angle downwards, we needed to elevate the tank. We wanted the tank to wobble slightly to add more water wave pattern, so we decided to attach hooks to the ceiling and to the corners of the tank, using four wires to suspend the tank. We then tested the height of the tank to see if it could support the light, and the speed of the drip unit, and the weighing of the tank. Finally, we fixed the drip unit to the tank.
Step 4:
Test the projection and adjust the projection position. Our project consisted of three projection walls. In our original plan, there were four sections to be projected, so Jin used madmapper to achieve a split screen and to solve the problem of distortion in the angled projection. However, we ended up eliminating a duplicate screen. As TD can hide components after zooming in, the software border is still present and affects the final visual effect. So we still intend to use Madmapper to solve this problem. Finally, we used syphonspoutout TOP to solve the connection problem between madmapper and touch designer.
Step 5:
As the walls in the room were rather messy, we hung curtains on the walls to ensure that the projection would be effective, making the walls clean. We wound various coloured threads, and these twisted colourful threads represent the influence of the outside world on people. They are like the various rules and potential deep impressions that bind us and will always be projected on our lives at some point.
At the same time, we have laid tinfoil underneath the water tank. We hope to create a richer light and visual effect through its reflective effect.
Step 6:
Adjust the angle of the light and test the water wave effect.
Step 7:
Adjust the position of the computer so that Max recognises the projected image in real time.
Final display effect:
Interpretation of the images on the three walls (software used):
Aqual Ripple consists of three projection walls and an installation. The three projection walls are: projection of the inner world, life from the front perspective and life from the side perspective.
The projection of the inner world is a sound visualisation created by the feedback function of TouchDesigner, which is based on audio generated by the microphone received by microphone and audio generated by MAX received by Visual Audio The audio generated by MAX received by Cable shows the corresponding frequency of sound and intensity through the shape of ripple.
The audience can stand in the room and interact by creating sounds, including singing, clapping, stomping, etc., and observing the corresponding changes produced by the digital ripple.
Front Life and Side Life are created by shooting two videos with the same content but at different angles, which are processed by MAX and received through the camera as content-filled inner projection content. The footage shows a fragment of a person’s life, from walking alone, rising up after being provoked, falling down, getting frustrated, being helped by a stranger, having a friend who spends time together…
The protagonist of the frontal life has a heart-shaped mask, and projections of the inner world are filled in as content in real time via the madmapper. Meanwhile, the entire character of the side life acts as a mask presenting a more pronounced inner world.
Show experience
The images are not only created by the vibrations and intensity of the music, but also by the microphone radios that change according to the sounds made by the audience. The whole experience is fun as the audience shouts, jumps, snaps their fingers, claps, sings and other interactive ways to make the images change even more.
The content of this blog consists of the work of Rudan and Jaela.
Rudan was responsible for the layout and visualisation of the poster, and Jaela for the drawing of the main character.
Rudan
The background image for the poster is from my friend Yiru, who gave it to me a few years ago.
I used a cut-out template to superimpose the background on the figures, the base colour of the figures in the top right corner is from the visual effect we created in Touch Designer; the base colour of the figures in the bottom left corner is from the build-up of different paints.
Regarding the design of the title. On the one hand I have cut the typeface and added some linear elements to make the title more abstract and add interest; on the other hand, the linear elements echo the water ripples and the lines representing the sensitive heart in our design concept. The liquefied black bar under the title, which I created using the liquefaction function in PS, softens the image while echoing the flow of the water.
Jaela
I abstracted the figures and drew the main figure in the line.
Reference
Hudson, D. (2019) Vibrant watercolour ink abstract background stock illustration – illustration of expressive, graphic: 163090094, Dreamstime. Available at: https://www.dreamstime.com/vibrant-watercolour-ink-abstract-background-digitally-created-watercolor-swirl-wash-texture-image163090094 (Accessed: April 26, 2023).
At the end of the test, we felt we needed to acquire some new props as well as update some of them to achieve better visual effects.
Water Tank
The size of the previous tank was too small, resulting in a black edge on the wall caused by the edge of the tank blocking the light when the water ripples were projected on the wall. Therefore, I decided to replace the water tank with an a3 size.
I purchased a 2mm thick sheet of 420 mm*297 mm clear PVC on eBay to make the bottom of the tank, and two 1mm thick sheets of 420 mm*297 mm clear PVC to make the four sides of the tank. I then cut these clear PVC sheets with a hobby knife, as well as using a glue gun to glue the five cut PVC sheets to make a sealed clear water tank.
Fortunately, the a3 size tank was just right for our needs. And the ripples in this tank project nicely on the wall.
Line
Considering that we wanted to show the complexity of emotions within people, I decided to use some thread to decorate the room for the exhibition. I purchased some fine sewing thread for hanging the water tank up, as it is only when the tank is at a certain height that the light can cast ripples through it onto the walls. I then brought out some of my hand-knitting threads in different colours, which intertwine in the room to create a better visual effect.
Bells
The bells can be used as a way for a person to interact with the device. The bells are attached to a wire and when a person touches the wire, the bells make sounds. These sounds can be blended perfectly with ambient sounds and can give the audience a sense of touch. In other words, the taut wire symbolises the sensitivity of the human heart, which is affected by the touch of others.
We re-shot the video of the people walking. The frontal walk as well as the side walk were filmed in multiple views.
I then imported the two videos into Capcut and used the video auto key feature to snap out the characters and make the character silhouettes black. After completing the two black character silhouette videos, I imported the videos into PR to make adjustments to both videos so that all the character actions in both videos were at the same point in time.
I added a background image to the side movement video of the characters, which was taken from an open source AI image. I tweaked and created that type of footage in PR to complete the background of the movement. The side video of the character was merged with this motion background to complete the side video.
These two videos will later be imported into Max and used to capture footage on location.
Heart keying
Create an alpha channel heart video that can correspond to the silhouette of a person in frontal movement, enabling live capture in Max.
Reference
Free vector: Parallax background with Autumn Forest and Lake (2021) Freepik. Available at: https://www.freepik.com/free-vector/parallax-background-with-autumn-forest-lake_21087491.htm#query=2d%20forest%20background&position=26&from_view=search&track=ais (Accessed: April 26, 2023).
Free vector: Silhouette shadow of forest scene (2022) Freepik. Available at: https://www.freepik.com/free-vector/silhouette-shadow-forest-scene_27282385.htm#page=2&query=2d%20forest%20background&position=18&from_view=search&track=ais (Accessed: April 26, 2023).
Pin on quick saves (2023) Pinterest. Available at: https://www.pinterest.jp/pin/710020697526232615/ (Accessed: April 26, 2023).
adding the amplitude of the music to make the connecting water tank ripples more visible and the changes in the water tank ripples more distinct with the rhythm of the music.
