The content of this blog consists of the work of Jin and Yuan.
______Jin______
There are two aspects I would love to talk about in this week’s blog. One is the space selection for our projection and installation, the other is effect enrichment.
During the past two weeks, we kept testing studio spaces and rooms to find a place where can achieve effects we want (sufficient projection distance and shading).
First we tested at E15A and E15B at ECA main building. Since the size of the room, it is hard for us to make the frame project fully on the wall. But in the positive aspect, they are dark enough and able to make the sound fill of the whole room.
We used TouchDesigner to make effects of colorful ripples. We found that when we place a collage created by PVC piece and laser paper or change the focus of projector to blur the picture, the final effect would be better combined with real-time water ripples.
simple device
Then we tried a photographic studio at Evolution House, where we tried to use camera to capture the overlay effect of TD virtual and real-time ripples. And then we made the capture picture as the content of the human mask (to represent the person’s internal emotions/heart).
The effect did not meet our expectations.
Also, when we projected the content of TouchDesigner, the software border cannot be hidden and it influences the audience’s experiences.
Therefore, I want to use Madmapper to control the frame size and shape the overlay effects.
First, I want to create mask in Madmapper, but it did not work. So through AE, I create a mask at the position of heart, which follows the movement of the person. Now the layer below can be seen through the mask.
I pre-recorded the TD ripples to test the effect and that went well. So I managed to make it a live input. I tried NDI camera to capture screen, but it didn’t work.
Then I found TouchDesigner has similar mapping function of Madmapper, so I tried the kantanMapper, but it has the same question that the full-screen function cannot work.
After that, I used syphon to connect TD and Madmapper. 
Then I tried to use the corner to show the inner world and external appearance.
—–yuan——
After these two weeks of constant adjustments and modifications, a version has now been formed. In terms of sound design, my initial idea was to make ambient music, but after testing it, I found that the sound was smooth and did not highlight the rhythm of the characters on screen。
so I added sound effects to the design. I used four samples as the ambient background sound. I used four samples for the ambient soundtrack, which were some of the water and ambient sounds we recorded outside last week, and then three dotted tones, using footsteps and water drops to make a soundscape. Also to match the image you need to take into account the rhythm, beat, pitch, timbre, mood and theme.
In our work, we show three sections of people walking, running and galloping to map the inner musical combination of the characters. Considering tha the movement of the person walking, running and galloping has a certain rhythm to it, matching the footsteps of the walk to the rhythm of the drums. Ambient sound effects are used to enhance the music. More relevant to the theme.
Combine Touch Designer’s audio visualization with the device’s water ripple:
In our tests we used the Touch Designer’s to pick up the audio from the max to make the water drops change. To create an effect that matches the rhythm and intensity of the audio.
