Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.
This week we test the effectiveness of the installation. The tests include the sound visualisation effect of max audio imported into Touch Designer, the effect of the installation’s water projection in combination with Touch Designer and the reworking of the projected image on the wall by max.
Based on the following flow chart, I was involved in optimising and practising the various sections. This includes testing in the field, optimisation of the effects of the installation, integration of the sound visualisation generated by Touch Designer with Max and editing of the Max video.
(This image was drawn by Rudan Zheng)
Wall One is used to representing the abstract inner world of the characters, using Touch Designer to recognise the audio to produce a sound visualisation.
Combining Touch Designer’s audio visualisation with the device’s water ripple:
To make the water ripples fit the projection better and to add colour to the ripples, I placed translucent sheets made of laser paper collage in front of the projector.
The effect of Wall 1 is as follows:
Wall II we use to show the real world of the characters and is therefore more figurative. The projection of wall one is recognised by Max, overlaid with the silhouette of the figure to create the image, and projected onto wall two. However, as the figure is too small and due to the distance captured by the computer camera, the actual effect is not as obvious as it would appear on the computer.
Philly gives us advice in the tutorial. On the one hand, the silhouettes of the people are too monotonous, on the other hand, we can try to capture the wall images with other equipment. So, we shot additional silhouettes of people. Based on the silhouette video we shot, I added a background to the video to increase the area available on Max for capturing the wall and to enrich the animation of the silhouette on the other hand.
The silhouettes of the people were generated in Max, and I created two different backgrounds for the scenes in Pr.
I combined the silhouette of the character with the sound visualisation generated by Touch Designer to achieve the following effect, which is also the desired one:
Put the figure in the max and combine it with the footage captured by the camera.
The content of this blog consists of the work of Jin and Yuan.
______Jin______
There are two aspects I would love to talk about in this week’s blog. One is the space selection for our projection and installation, the other is effect enrichment.
During the past two weeks, we kept testing studio spaces and rooms to find a place where can achieve effects we want (sufficient projection distance and shading).
First we tested at E15A and E15B at ECA main building. Since the size of the room, it is hard for us to make the frame project fully on the wall. But in the positive aspect, they are dark enough and able to make the sound fill of the whole room.
We used TouchDesigner to make effects of colorful ripples. We found that when we place a collage created by PVC piece and laser paper or change the focus of projector to blur the picture, the final effect would be better combined with real-time water ripples.
simple device
Then we tried a photographic studio at Evolution House, where we tried to use camera to capture the overlay effect of TD virtual and real-time ripples. And then we made the capture picture as the content of the human mask (to represent the person’s internal emotions/heart).
The effect did not meet our expectations.
Also, when we projected the content of TouchDesigner, the software border cannot be hidden and it influences the audience’s experiences.
Therefore, I want to use Madmapper to control the frame size and shape the overlay effects.
First, I want to create mask in Madmapper, but it did not work. So through AE, I create a mask at the position of heart, which follows the movement of the person. Now the layer below can be seen through the mask.
I pre-recorded the TD ripples to test the effect and that went well. So I managed to make it a live input. I tried NDI camera to capture screen, but it didn’t work.
Then I found TouchDesigner has similar mapping function of Madmapper, so I tried the kantanMapper, but it has the same question that the full-screen function cannot work.
After that, I used syphon to connect TD and Madmapper.
Then I tried to use the corner to show the inner world and external appearance.
—–yuan——
After these two weeks of constant adjustments and modifications, a version has now been formed. In terms of sound design, my initial idea was to make ambient music, but after testing it, I found that the sound was smooth and did not highlight the rhythm of the characters on screen。
so I added sound effects to the design. I used four samples as the ambient background sound. I used four samples for the ambient soundtrack, which were some of the water and ambient sounds we recorded outside last week, and then three dotted tones, using footsteps and water drops to make a soundscape. Also to match the image you need to take into account the rhythm, beat, pitch, timbre, mood and theme.
In our work, we show three sections of people walking, running and galloping to map the inner musical combination of the characters. Considering tha the movement of the person walking, running and galloping has a certain rhythm to it, matching the footsteps of the walk to the rhythm of the drums. Ambient sound effects are used to enhance the music. More relevant to the theme.
Combine Touch Designer’s audio visualization with the device’s water ripple:
In our tests we used the Touch Designer’s to pick up the audio from the max to make the water drops change. To create an effect that matches the rhythm and intensity of the audio.
Based on the content of this week's Jules tutorial and Jules' advice to us, we have made further improvements to our design solution. By using Touch Designer and Max, we have increased the interactivity of the design and improved the visual effect of the design. Rudan and Jin were responsible for the part of Touch Designer, and Jeala and Yuan for the part of Max.
Design Development
Our design concept is a ‘human lifetime’ and the implementation is still water, sound and installation. But we wanted to achieve the effect of ‘audience-action/sound-installation-visuals’ interacting with each other and changing in real-time with the audience’s movement/sound, so we introduced Touch Designer and Max. The interaction flow is shown in the diagram below:
(This image was drawn by Rudan Zheng)
We plan to show different effects on the two walls, one wall shows people’s inner world, and the other wall shows people’s behavior and actions. These two walls are displayed together with the installation and the projector screen, which can interact with the audience in real-time.
It can be understood that we are showing a “person”, and the voice and behavior of every audience who comes to watch this “person” can have an impact on this person, and this influence will change the shape of the projection on the wall.
Through learning some TD tutorials on Youtube and testing, we achieved the visualisation of sound in real-time. The input can be translated into flowing water, and changed based on the rhythm and strength.
Rudan’s First Test
Jin’s Second Version①
Jin’s Second Version②
I used VB-audio Virtual Cable as the input and output of sounds. Therefore we can change the music at any time, just change the music in spotify or youtube, without inputing music files into Touch Designer and use switch to change. Also, it can deal with sound from microphone, environment sounds and music in place and generate animations, which we would like to project on the water ripples created by our installation.
Besides, the colors and shapes of the water ripples in TD can be changed through ramp and displace components respectively (details can be found in video and pictures①②). The size of ripples can be changed by paramater multiply in maths component. The volume of music can also be flexible to adjust.
One obvious problem is the quality of sound, and until now I have no idea about how to improve this.
The content of this blog consists of the work of Jaela and Yuan.
Jaela and Yuan
With the feedback received last week, tests were carried out this week with different materials and different models of sound.
VibrationTest
The first sound vibration test was carried out with the Subwoofer.
Test 1: We glued the pvc sheets together to form a square container. A suitable amount of water was poured in for the vibration test. The water only slightly produced some vibrations.
Reflection: The container was too thick and heavy for the vibrations of the sound to travel up and produce vibrations easily。
Test 2:A test using a plastic disc to produce vibrations. The water waves were very small and the overall effect was still not good.
Reflection: The container may still be too thick and heavy and needs to be replaced with a lighter container.
Test 3: A very thin plastic cup was used for the vibration test. The vibration was very strong and the water waves were obvious.
Reflection: The material of the container needs to be very thin and light so that the vibration will be strong.
Test 4: Sound vibration test with Subwoofer and Genelec-8040-speaker respectively. Using the transparent plastic plate, the vibrations are strong and the water waves are large. But splashes of water may affect the projection effect.
Reflection: The amount of water needs to be controlled, too much will diminish the vibrations, too little will produce a lot of splashing water droplets.
Test 5:Tested with a plastic bag, the vibration effect is good, but the material is too soft and deforms when water is poured in.
Reflection: The material is also too soft and may affect the projection effect. The choice of material is important in order not to affect the projection effect.
Original plan：
The speaker is placed flat under a table with a container on top of it, on the diaphragm of the speaker, and water drops are placed in the container so that the drops create ripples through the sonic Hertzian vibrations of the music. We will take photographs of the water’s shadow and light and record some sounds. These photographs and sounds will be learned by the AI, such as Mubert-text-to-music, disco diffusion, and clip interrogator, and then the AI will produce some musical compositions. Before each stage begins, we invite the audience to pick a piece of music from the three sections we provide to influence the ripples.
Modified programme：
We had a meeting to discuss our project. The original plan was to have three stages of music, a lot of pieces, but now we want to cut out the extra pieces,we can do two or three different kind of concepts and changes, Locations that kind of reflect different aspects of the journey of life right. The presentation of the movement of the characters in the picture is changed through different musical rhythms. The musical rhythm of a person walking can be steady and even. The musical rhythm of a runner should be faster than that of a walker and can be performed with a faster tempo. The musical rhythm of a fall is usually more chaotic and can be expressed by using irregular beats or by using some techniques such as staccato and pauses in the music. Life is like a long run, and we need to run persistently to achieve success. In the process of running, we may experience various difficulties and challenges, but as long as we always persist in our belief of moving forward, we will eventually reach the end of success.
we can tie into the distance sensor and use the distance sensor as a kind of responsive tool as well for the music and turn that up and down or maybe how proximity as away ，and Music .we initially intended that we could use artificial intelligence to generate audio. But then the opportunity to create dynamic, responsive work would be lost. So we would use max. and we wanted the end result to be, like
The gap between the stereo and the container can only hold small bulbs. The effect of putting a small bulb under the container to cast is not very obvious. Perhaps we could make use of the water wave footage that was recorded in advance. By projecting this footage to the ai, the scene could be projected with some of the patterns generated by the ai. Two walls could be used, one projecting the projection of the installation and the other projecting the ai generated effect. It might be more interesting for the audience to experience two different effects at the same time by touching or sensing to make the projection and the music change.
Recording equipment
l material selection
l High sample rate
l High audio quality
Recording Programme A.
Sound is a very important form of expression in a person’s life, and it can record a person’s growth from infancy to old age. It can be a valuable way of recording, as well as reminding people of past events and reflecting the preciousness and profundity of life.
Infancy: the sound of a baby crying, lullabies in a cot, parents kissing, the sound of a warm family atmosphere.
Adolescence: the sound of classes in school, the sound of activities on campus, the sound of students communicating, the sound of themselves at play, the sound of television or films.
Middle age: ambient sounds at work, conversations in the home, children playing, sounds of family activities, the sound of music when you are relaxing and resting.
Old age: sounds of family life, the company of family members, walking in the street, music while relaxing, sounds of natural landscapes.
Site reference.
Schools; meadows; sounds of people walking in the street
Recording Programme B:
Record the sounds of the natural environment at different times of the day or in different weather conditions, such as the sounds of nature, water currents, the sound of the wind on the beach, etc. These natural sounds can be recorded at different stages of life and in different situations, and can be poured into the max to create music that resonates with the audience.
Reflection
I bought a separate amplifier unit, soldered and assembled it, and ran a new test on Sunday as the previous test had not worked well.
The locations chosen for this recording were the Royal Botanic Gardens in Edinburgh and portobrllo beach, where the ambient sound was recorded. We capture the ambient sound as a basis for the music and edit it in max to create an ambient music. max is obviously the most important direction we should work on.
The microphone of choice is the Sennheiser – MKH60 – Shotgun – Hypercardioid, as hypercardioid microphones are often used when you need to pick up sound in noisy environments to accentuate your voice while reducing the effect of ambient noise on the recording. Ideal for recording ambient sound. Using a portable digital recorder, we chose a location close to a stream where there was not too much background noise. After continually adjusting the direction and height of the microphone, we found a location with better sound reception to better capture the sound of the water flow. The recording was edited and adjusted using protools.
After the group discussion, we had a preliminary distribution of work. My job was the preparation and purchase of materials.
Lights
First I tried a mobile phone light source as a light for the water ripple reflection and determined that a point light source was the best light for reflecting the ripples.
I then tried a larger yellow spherical light source and found that this light source, because of its particular construction and colour, reflected colourful light through the object on the wall.
I tried other light sources of different sizes, such as a circular torch, a long strip of light, and a small torch.
But none of the results obtained were as good as the mobile phone light source and the yellow spherical light source. I presume this is related to the intensity and size of the light source.
So, after a discussion with the rest of the team, I decided to go for a mobile phone light and a yellow sphere light. The final choice depends on the final result.
Water Tank
Considering the light transmission of the water tank, I preferred to use glass as the material of the water tank. But as can be seen in Figures 1 and 2, whether it is colourless or tinted glass, it is difficult to reflect the ripples of the water visibly on the wall as the glass is too thick and has a strong light reflection of its own. So I came up with the idea of a transparent PVC material.
First, I purchased 110mm*110mm*110mm pieces of clear PVC and then glued them together with a glue gun to form a square water tank. Through testing, I found that the transparent PVC material had a good weight and light transmission, which was very much in line with our project requirements.
However, in subsequent tests I found that the tank was too small, resulting in a small projection area on the wall and a noticeable black edge. I, therefore, intend to re-purchase the clear PVC.
Laser Paper
In comparing the projection of the PVC water tank with a mobile phone light source and a yellow spherical light source, we have tentatively decided on a mobile phone light source. In order to achieve the light reflection in our design concept, we wanted to use laser paper to achieve this effect. This means that the white light source is reflected through the laser paper, causing the light to reflect in different colours on the wall.
However, practical tests showed that the projection of light on the wall through the single laser paper was not obvious.
Transparent PVC SheetS with Pigments
As the projection of the single laser paper on the wall was not obvious, I tried cutting the laser paper and pasting it in the form of a collage on a transparent PVC sheet. But the projection was also mediocre.
Then I tried to colour the transparent PVC sheet with gouache paint so that the light would be projected on the wall through the coloured translucent PVC sheet. However, the projection was still not good.
Therefore, I am trying to find other ways to make the light appear colourful on the wall.
Drip Device
We were going to make our own drip device but I suggested that we could just buy medical drip props which would give us better control of the drip rate. As medical drip props are not sold in the UK, not even for plants. I, therefore, decided to purchase them from China and have them couriered to Edinburgh. As the time between purchase and receipt was greater than two weeks and returns were not possible, I opted to purchase two different drip props for plants.
Arduino
We expect the Arduino to control the start of the drip unit, the running of the unit and the projection of the projector.
After receiving feedback from Philly on our sound design, we decided to broaden our understanding of audio-visual installation pieces and consider the following aspects:
Immersive experience: Our goal is to create sound environments that deeply engage the viewer and provide a unique experience, different from traditional visual art.
1. Creative experimentation: We are interested in exploring the possibilities of audio processing, sound structure, and spatial design to push the boundaries of traditional music and sound art and create novel sonic experiences.
2. Multiculturalism: We aim to cross-cultural and linguistic barriers and break genre restrictions by fusing different sound and music elements, achieving cross-cultural communication and integration.
3. Social reflection: We recognize the power of sound in drawing attention to and reflecting on social issues. We hope to call for social concern and change by creating socially significant works.
4. Environmental consciousness: We are interested in using sound materials from the environment to explore the sound properties and spatial nature of natural and man-made environments. Through our work, we hope to promote environmental protection and sustainable development.
Sound installation art is a unique art form that uses sound and music as its primary medium to provoke sensory and intellectual experiences in the viewer. It explores sound’s different properties and manifestations, leading to a deep and immersive artistic experience.
Project One: “Towards the reproduction of sound”
In the video Mediation: Towards the Reproduction of Sound, Hill starts sprinkling rice on a speaker and then speaks through the microphone himself at the same time, his voice emanating from the speaker’s loudspeaker, and the grains of rice bounce along with the sound waves, with different vibrations of rice in both high and low tones, as if they were many small intelligent creatures dancing as a whole under the direction of the artist’s words. Towards the end of the video, the rice grains completely bury the speakers of the speaker, covering the sound.
In this work, Hill is burying sound while talking to himself. The language of sound that he emits through the speakers actually describes this visualisation process. He uses the simplest of words to say things like “sound, speaker, hand”, and then these simple images of words appear to the viewer. Hill uses this experiment to show the audience that sound as a substance is real and can be seen and touched as the speakers you see in front of you and the artist’s hands.
“Sonic Water” is an interactive installation experiment by KYMAT Laboratories in Germany that studies the visualization and vibration of sound and their ability to express sound aesthetically, transforming it into a fascinating visual experience.
Sonic Water is unique in its form and presentation as a sound installation. The artist has created a distinctive sound space and experience by combining water flow and sound. The work is also highly interactive, allowing viewers to alter the sound and form of the work through gestures and movements.
In terms of reflection on water resources, Sonic Water’s creative process involves using a significant amount of water resources. As we become increasingly concerned about environmental protection and sustainable development, it is essential to reflect on how to better utilise and protect water resources. Moreover, we should focus on how to use the process of creating artwork to call for environmental protection and promote awareness about water conservation.
Reflection：
For the sound installation of water droplets, focus on using the characteristics of the exhibition space and environment to build a more intimate and deeper relationship with the audience through the experience and perception of sound. The above similar works also make use of sound vibrations. Sound installation art is an art form and a process of creation and exploration. Using the sound of water droplets to stimulate human perception to create resonance and emotional resonance. By creating an organic sound space, the viewer can experience the power and emotion of sound through the auditory sensation of physical change. The work is driven and changed by the actions and reactions of the audience.
– MAXMSP-
MAXMSP is a powerful music creation and performance software that allows users to create various music and audio applications through a visual programming language. Learning MAXMSP requires some programming experience and musical knowledge. Although MAXMSP’s visual programming language is very intuitive, understanding basic programming concepts and musical principles is helpful in learning MAXMSP. This was a challenge for our group, but we were very interested.
This week we have learnt the basic MAX objects and operations and tried and practised a small part of them. And experimented with some simple linked music.
The blog content is by Rudan in collaboration with Jin.
Blog Background:
According to Philly's feedback about our submission one, we tend to reflect our initial ideas and concepts and analyze other installation projects, in order to have a deeper understanding of wider context.
In addition, after discussion, our group is divided into two sub groups for further research and clear division of labor. Rudan and Jin are responsible for more detailed Arduino coding and installation creating process. Jalea and Yuan are in charge of music generation with MAX. In this blog, I (Jin) and Rudan would make a reflection on our submission one based on our research on other installations with Arduino and lights.
Plants are often overlooked, but they live in symbiosis with us. They collect invisible information about nature and express it through their own state – the color of their leaves is richly colored, their form is uplifted or drooping – and they are the most important part of our lives.
This project aims to explore the possibilities of human-plant interaction by ‘touching’ plants through art installations and ‘observing’ their data – light, water, and emotion. This will help people to understand what plants know and feel, and to deepen their connection with them.
【Notes: There is a certain similarity to our concept. The design uses the Arduino and the installation to show the life of a plant, and our design uses water to show the life of a person. Through this design, I thought about the possibility of using devices to complement the concept of ‘human life’.】
Studyable component:
The way in which touch sensors and humidity sensors pick up data and import it into the device (data transfer from the device via Arduino only).
The way in which the plant state is represented figuratively.
The way in which the device interacts with people.
Arduino:
The installation is divided into two parts: the moss and the device (blue optical fiber). The moisture sensor is used to obtain soil moisture data, which is fed into the Arduino, and the blue light dot on the optical fiber increases. In this way, the moisture of the soil is visualized.
By means of a touch sensor, the red light spots on the optical fibers increase when a person touches the moss.
【Notes: The installation visualizes the state of the plant in a fiber-optic variation. In our design, the concept of the ‘human life cycle’ was intended to be presented through the changing projection of water on the wall, but this did not seem to be a direct way of making the viewer appreciate our intentions. We thought about the possibility of using some sensors to make the projection of the water change and at the same time make the wall appear to represent the stages of a person’s life (child, middle-aged, elderly).】
The materials are mirrors, acrylic panels, acrylic columns, UV adhesive, and the model is purely handmade. A visual art display combined with the materials.
【Notes: The sound and nature of the material can be used wisely to achieve unexpected visual effects.】
Studyable component:
Visual effects.
Sound effects.
【Notes: Consider whether the sounds and effects of the material can be used to further highlight the changing ‘stages of life’.】
– Thinking about the development –
Using the principle of pin-hole image, the projection of ‘a life of a human’ on the wall will change.
For example, in our design we use wall projections to present the three stages of a person’s life, i.e. in the first stage, the projection of a child appears on the wall and the colorful ripples produced by the water surface are reflected in the child’s projection; in the second stage, the projection switches to a middle-aged person and the colorful ripples reflected in the middle-aged person’s projection change; in the third stage, the projection switches to an elderly person and the ripples change with the music.
(This image was drawn by Rudan Zheng)
Transformation of pictures in small-aperture imaging using the kaleidoscope principle, achieved with a mechanical gear mechanism.
Conception: the weight of the water is sensed by a pressure sensor and the rotation of the gear is controlled by the change in weight (the fall of the steel ball). The picture of the small hole imaged is bound to the gear and changes when the mechanical gear undergoes rotation. Case reference:
Combine fragments of highly reflective materials such as broken mirrors and glass with the reflection of light to make it more complex and achieve a more striking visual effect.
Consider incorporating the sound of breaking mirrors and glass into the soundtrack of the ‘middle age’ stage to convey the idea that life is not always easy.
At the end of the first session, all the members of Process discussed and decided to work in two groups. I decided to form a group with Jin Qin, Yuan Tian and Jeala Wang.
Before the first group discussion, I proposed that we each come up with some options and then pick the most interesting one from our options during the group discussion and use that as the basis to start making our project.
I had discussed with Philly in class about the water, light and ripple design and I thought it was interesting and cool and the idea was approved by Philly. So I tried to use water, light and ripples as elements and came up with the following options:
Sounds of the seasons: capturing the sounds of the seasons and synthesising them into a piece of music using artificial intelligence
A colour-blind world: If I am colour-blind, can I experience a colourful world through sound
Musical Puzzles: exploring whether people who don’t know music can make music through collage (similar to a material puzzle, then with an ai music synthesiser)
Music visualisation installation: figurative music
Vibrations of music (amplified) – water/vibrations of lines/line drawings/other media
Music and Time: Rhythm and the different frequencies of time, exploring whether music can change one’s perception of time
Astigmatic world: what would the world look like if I blurred your eyes (using ai as an example, blurring images with woolen glass or water and analyzing with ai)
References
– light art – light installations – reflections and shadows: Light art installation, light art, light installation (2023) Pinterest. Available at: https://www.pinterest.jp/pin/731483164496784888/ (Accessed: February 10, 2023).
Ellen Barratt – light work: Light art installation, projection installation, light installation (2023) Pinterest. Available at: https://www.pinterest.jp/pin/731483164496686680/ (Accessed: February 10, 2023).
Momento wonderglass (2023) Pinterest. Available at: https://www.pinterest.jp/pin/731483164496784817/ (Accessed: February 10, 2023).
Pin On Water (2023) Pinterest. Available at: https://www.pinterest.jp/pin/731483164496784814/ (Accessed: February 10, 2023).
‘under the barbers’ shop’ by Ellen Barratt – 2017, London – Interactive Light Installation: Light Art Installation, light installation, artistic installation (2023) Pinterest. Available at: https://www.pinterest.jp/pin/731483164496686762/ (Accessed: February 10, 2023).
simple device
In our work, we show three sections of people walking, running and galloping to map the inner musical combination of the characters. Considering tha the movement of the person walking, running and galloping has a certain rhythm to it, matching the footsteps of the walk to the rhythm of the drums. Ambient sound effects are used to enhance the music. More relevant to the theme.
Therefore, I am trying to find other ways to make the light appear colourful on the wall.
