Any views expressed within media held on this service are those of the contributors, should not be taken as approved or endorsed by the University, and do not necessarily reflect the views of the University in respect of any particular issue.

Group 2_Character Movement Animation & Pr

Character Movement Animation

We re-shot the video of the people walking. The frontal walk as well as the side walk were filmed in multiple views.

I then imported the two videos into Capcut and used the video auto key feature to snap out the characters and make the character silhouettes black. After completing the two black character silhouette videos, I imported the videos into PR to make adjustments to both videos so that all the character actions in both videos were at the same point in time.

I added a background image to the side movement video of the characters, which was taken from an open source AI image. I tweaked and created that type of footage in PR to complete the background of the movement. The side video of the character was merged with this motion background to complete the side video.

These two videos will later be imported into Max and used to capture footage on location.

Heart keying

Create an alpha channel heart video that can correspond to the silhouette of a person in frontal movement, enabling live capture in Max.

Reference

Free vector: Parallax background with Autumn Forest and Lake (2021) Freepik. Available at: https://www.freepik.com/free-vector/parallax-background-with-autumn-forest-lake_21087491.htm#query=2d%20forest%20background&position=26&from_view=search&track=ais (Accessed: April 26, 2023).

Free vector: Silhouette shadow of forest scene (2022) Freepik. Available at: https://www.freepik.com/free-vector/silhouette-shadow-forest-scene_27282385.htm#page=2&query=2d%20forest%20background&position=18&from_view=search&track=ais (Accessed: April 26, 2023).

Pin on quick saves (2023) Pinterest. Available at: https://www.pinterest.jp/pin/710020697526232615/ (Accessed: April 26, 2023).

Group 2_Character Movement Animation-Video capture&Video Editing

Character Movement Animation Video capture

Because before I only shot a video of a person moving, but the projection effect was not ideal too monotonous. Philly suggested we make it more graphic. So we decided to re-shoot some of the characters and cut them together.

 

I use the button function in the editing software to select the text. Change the background to white. By adjusting the brightness of the curve, the figure becomes a black silhouette. After editing, I drag and drop this video into Max and apply it.

Group 2_Installation Performance, Max, Pr

Rudan

This week we test the effectiveness of the installation. The tests include the sound visualisation effect of max audio imported into Touch Designer, the effect of the installation’s water projection in combination with Touch Designer and the reworking of the projected image on the wall by max.

Based on the following flow chart, I was involved in optimising and practising the various sections. This includes testing in the field, optimisation of the effects of the installation, integration of the sound visualisation generated by Touch Designer with Max and editing of the Max video.

(This image was drawn by Rudan Zheng)

Wall One is used to representing the abstract inner world of the characters, using Touch Designer to recognise the audio to produce a sound visualisation.

Combining Touch Designer’s audio visualisation with the device’s water ripple:

To make the water ripples fit the projection better and to add colour to the ripples, I placed translucent sheets made of laser paper collage in front of the projector.

The effect of Wall 1 is as follows:

Wall II we use to show the real world of the characters and is therefore more figurative. The projection of wall one is recognised by Max, overlaid with the silhouette of the figure to create the image, and projected onto wall two. However, as the figure is too small and due to the distance captured by the computer camera, the actual effect is not as obvious as it would appear on the computer.

Philly gives us advice in the tutorial. On the one hand, the silhouettes of the people are too monotonous, on the other hand, we can try to capture the wall images with other equipment. So, we shot additional silhouettes of people. Based on the silhouette video we shot, I added a background to the video to increase the area available on Max for capturing the wall and to enrich the animation of the silhouette on the other hand.

The silhouettes of the people were generated in Max, and I created two different backgrounds for the scenes in Pr.

I combined the silhouette of the character with the sound visualisation generated by Touch Designer to achieve the following effect, which is also the desired one:

 

Put the figure in the max and combine it with the footage captured by the camera.

 

 

 

 

Group 2_Teamwork_Mar 29_TouchDesigner, AE and MadMapper

The content of this blog consists of the work of  Jin and Yuan.

______Jin______

There are two aspects I would love to talk about in this week’s blog. One is the space selection for our projection and installation, the other is effect enrichment.

During the past two weeks, we kept testing studio spaces and rooms to find a place where can achieve effects we want (sufficient projection distance and shading).

First we tested at E15A and E15B at ECA main building. Since the size of the room, it is hard for us to make the frame project fully on the wall. But in the positive aspect, they are dark enough and able to make the sound fill of the whole room.

We used TouchDesigner to make effects of colorful ripples. We found that when we place a collage created by PVC piece and laser paper or change the focus of projector to blur the picture, the final effect would be better combined with real-time water ripples.

simple device

Then we tried a photographic studio at Evolution House, where we tried to use camera to capture the overlay effect of TD virtual and real-time ripples. And then we made the capture picture as the content of the human mask (to represent the person’s internal emotions/heart).

The effect did not meet our expectations.

Also, when we projected the content of TouchDesigner, the software border cannot be hidden and it influences the audience’s experiences.

Therefore, I want to use Madmapper to control the frame size and shape the overlay effects.

First, I want to create mask in Madmapper, but it did not work. So through AE, I create a mask at the position of heart, which follows the movement of the person. Now the layer below can be seen through the mask.

I pre-recorded the TD ripples to test the effect and that went well. So I managed to make it a live input. I tried NDI camera to capture screen, but it didn’t work.

Then I found TouchDesigner has similar mapping function of Madmapper, so I tried the kantanMapper, but it has the same question that the full-screen function cannot work.

After that, I used syphon to connect TD and Madmapper. 

Then I tried to use the corner to show the inner world and external appearance.

—–yuan——

After these two weeks of constant adjustments and modifications, a version has now been formed. In terms of sound design, my initial idea was to make ambient music, but after testing it, I found that the sound was smooth and did not highlight the rhythm of the characters on screen。

屏幕录制2023-03-30 01.44.32

so I added sound effects to the design. I used four samples as the ambient background sound. I used four samples for the ambient soundtrack, which were some of the water and ambient sounds we recorded outside last week, and then three dotted tones, using footsteps and water drops to make a soundscape. Also to match the image you need to take into account the rhythm, beat, pitch, timbre, mood and theme.

In our work, we show three sections of people walking, running and galloping to map the inner musical combination of the characters. Considering tha the movement of the person walking, running and galloping has a certain rhythm to it, matching the footsteps of the walk to the rhythm of the drums. Ambient sound effects are used to enhance the music. More relevant to the theme.

Combine Touch Designer’s audio visualization with the device’s water ripple:

In our tests we used the Touch Designer’s to pick up the audio from the max to make the water drops change. To create an effect that matches the rhythm and intensity of the audio.

Group 1 – Interactivity Tablet Tests

In the ever-evolving digital world, interactivity plays a pivotal role in captivating audiences and crafting unforgettable experiences. As tablets continue to gain traction as a popular interactive medium, understanding how to effectively test and implement these elements is paramount.

Resolume Arena

Resolume Arena is a formidable VJ software that empowers users to create live visuals, projections, and interactive experiences. Its comprehensive features make it the perfect choice for developing engaging content tailored to any audience. By incorporating Resolume Arena with a tablet, we can offer our audience the opportunity to interact with the visuals in real-time, providing an immersive experience that can be adapted to any event or installation.

OSC/Pilot

OSC/Pilot is an adaptable control software that enables users to craft custom interfaces for managing various applications, including Resolume Arena. When used in conjunction with a tablet, OSC/Pilot becomes an outstanding tool for facilitating public interaction with our visuals. However, we encountered a challenge: OSC/Pilot requires a license to save, load and modify workspaces directly on a tablet.

To overcome the free licensing restrictions associated with editing OSC/Pilot workspaces on a tablet, we found a clever solution: TeamViewer. By installing TeamViewer on both our laptop and tablet, we can remotely access and edit the workspaces on the tablet without the need for an OSC/Pilot license.

Here’s how we made it work:

  1. Install TeamViewer on both our laptop and tablet.
  2. Establish a connection between the devices by entering the TeamViewer ID and password.
  3. Connect Resolume Arena OSC controls with OSC/Pilot
  4. Once connected, remotely access our laptop’s screen from the tablet, allowing us to edit OSC/Pilot workspaces as if we were working directly on the laptop.

Edit: Since Resolume Arena shows a watermark when importing own video, we decided to use TouchDesigner along with the OSC/Pilot for controls.

TouchDesigner

TouchDesigner is a versatile visual programming environment that enables artists, designers, and developers to create real-time interactive multimedia projects. With a node-based interface, it allows users to quickly build complex visuals by connecting various components, making it perfect for live performances, installations, and interactive experiences.

This workaround has been instrumental in providing us the flexibility to create engaging interactive experience.

 

Test setup & videos:

Audio recording

Recording equipment

 

  • l material selection
  • l High sample rate
  • l High audio quality

Recording Programme A.

Sound is a very important form of expression in a person’s life, and it can record a person’s growth from infancy to old age. It can be a valuable way of recording, as well as reminding people of past events and reflecting the preciousness and profundity of life.

  1. Infancy: the sound of a baby crying, lullabies in a cot, parents kissing, the sound of a warm family atmosphere.
  2. Adolescence: the sound of classes in school, the sound of activities on campus, the sound of students communicating, the sound of themselves at play, the sound of television or films.
  3. Middle age: ambient sounds at work, conversations in the home, children playing, sounds of family activities, the sound of music when you are relaxing and resting.
  4. Old age: sounds of family life, the company of family members, walking in the street, music while relaxing, sounds of natural landscapes.

 

Site reference.

Schools; meadows; sounds of people walking in the street

 

Recording Programme B:

Record the sounds of the natural environment at different times of the day or in different weather conditions, such as the sounds of nature, water currents, the sound of the wind on the beach, etc. These natural sounds can be recorded at different stages of life and in different situations, and can be poured into the max to create music that resonates with the audience.

 

Reflection

I bought a separate amplifier unit, soldered and assembled it, and ran a new test on Sunday as the previous test had not worked well.

 

The locations chosen for this recording were the Royal Botanic Gardens in Edinburgh and portobrllo beach, where the ambient sound was recorded. We capture the ambient sound as a basis for the music and edit it in max to create an ambient music. max is obviously the most important direction we should work on.

 

The microphone of choice is the Sennheiser – MKH60 – Shotgun – Hypercardioid, as hypercardioid microphones are often used when you need to pick up sound in noisy environments to accentuate your voice while reducing the effect of ambient noise on the recording. Ideal for recording ambient sound. Using a portable digital recorder, we chose a location close to a stream where there was not too much background noise. After continually adjusting the direction and height of the microphone, we found a location with better sound reception to better capture the sound of the water flow. The recording was edited and adjusted using protools.

 

 

 

 

 

 

 

 

 

 

 

Group2-Jaela-Max video development

After the development meeting with Philly, Max video made some progress. Because what I want is for the real time content of the screen to only be displayed in silhouette. Philly told me that CHROMAKEYR might do the trick. I started trying, but it didn’t work.

Through continuous attempts, I finally found that I could pass the Operation! – This parameter achieves the effect I was after.

Group2-Jaela-Max video

Jaela_Max

Reference:

Final effect:

Let MAX capture the scene of the water wave screen to fill the character content, the outer outline is black. This silhouette is the external image of a person.

Test 1:

In test one I made the projection and then added the live interactive camera. Give them some interesting colors and effects.

Problem:

I don’t know how to control a live projection with just silhouettes. The only thing I can do is project the whole screen in real-time. I wonder if Max can do this. If that doesn’t work maybe I can do it in a different way.

Vibration Test

The first sound vibration test was carried out with the Subwoofer.

 

Test 1: We glued the pvc sheets together to form a square container. A suitable amount of water was poured in for the vibration test. The water only slightly produced some vibrations.

Reflection: The container was too thick and heavy for the vibrations of the sound to travel up and produce vibrations easily。

 

Test 2:A test using a plastic disc to produce vibrations. The water waves were very small and the overall effect was still not good.

Reflection: The container may still be too thick and heavy and needs to be replaced with a lighter container.

 

Test 3: A very thin plastic cup was used for the vibration test. The vibration was very strong and the water waves were obvious.

Reflection: The material of the container needs to be very thin and light so that the vibration will be strong.

Test 4: Sound vibration test with Subwoofer and Genelec-8040-speaker respectively. Using the transparent plastic plate, the vibrations are strong and the water waves are large. But splashes of water may affect the projection effect.

Reflection: The amount of water needs to be controlled, too much will diminish the vibrations, too little will produce a lot of splashing water droplets.

Test 5:Tested with a plastic bag, the vibration effect is good, but the material is too soft and deforms when water is poured in.

Reflection: The material is also too soft and may affect the projection effect. The choice of material is important in order not to affect the projection effect.

Original plan

The speaker is placed flat under a table with a container on top of it, on the diaphragm of the speaker, and water drops are placed in the container so that the drops create ripples through the sonic Hertzian vibrations of the music. We will take photographs of the water’s shadow and light and record some sounds. These photographs and sounds will be learned by the AI, such as Mubert-text-to-music, disco diffusion, and clip interrogator, and then the AI will produce some musical compositions. Before each stage begins, we invite the audience to pick a piece of music from the three sections we provide to influence the ripples.

Modified programme

We had a meeting to discuss our project. The original plan was to have three stages of music, a lot of pieces, but now we want to cut out the extra pieces,we can do two or three different kind of concepts and changes, Locations that kind of reflect different aspects of the journey of life right. The presentation of the movement of the characters in the picture is changed through different musical rhythms. The musical rhythm of a person walking can be steady and even. The musical rhythm of a runner should be faster than that of a walker and can be performed with a faster tempo. The musical rhythm of a fall is usually more chaotic and can be expressed by using irregular beats or by using some techniques such as staccato and pauses in the music. Life is like a long run, and we need to run persistently to achieve success. In the process of running, we may experience various difficulties and challenges, but as long as we always persist in our belief of moving forward, we will eventually reach the end of success.

 

we can tie into the distance sensor and use the distance sensor as a kind of responsive tool as well for the music and turn that up and down or maybe how proximity as away ,and Music .we initially intended that we could use artificial intelligence to generate audio. But then the opportunity to create dynamic, responsive work would be lost. So we would use max. and we wanted the end result to be, like

https://www.bilibili.com/video/BV1Si4y197Tx/?spm_id_from=333.337.search-card.all.click&vd_source=43a1873fafb51f0936b53c75c77f14f3

 Installation Reflection.

 The gap between the stereo and the container can only hold small bulbs. The effect of putting a small bulb under the container to cast is not very obvious. Perhaps we could make use of the water wave footage that was recorded in advance. By projecting this footage to the ai, the scene could be projected with some of the patterns generated by the ai. Two walls could be used, one projecting the projection of the installation and the other projecting the ai generated effect. It might be more interesting for the audience to experience two different effects at the same time by touching or sensing to make the projection and the music change.

 

 

Group 2_Teamwork_Jin&Rudan_Mar 19_Touch Designer

Week8

Based on the content of this week's Jules tutorial and Jules' advice to us, we have made further improvements to our design solution. By using Touch Designer and Max, we have increased the interactivity of the design and improved the visual effect of the design. Rudan and Jin were responsible for the part of Touch Designer, and Jeala and Yuan for the part of Max.

Design Development

Our design concept is a ‘human lifetime’ and the implementation is still water, sound and installation. But we wanted to achieve the effect of ‘audience-action/sound-installation-visuals’ interacting with each other and changing in real-time with the audience’s movement/sound, so we introduced Touch Designer and Max. The interaction flow is shown in the diagram below:

(This image was drawn by Rudan Zheng)

We plan to show different effects on the two walls, one wall shows people’s inner world, and the other wall shows people’s behavior and actions. These two walls are displayed together with the installation and the projector screen, which can interact with the audience in real-time.

It can be understood that we are showing a “person”, and the voice and behavior of every audience who comes to watch this “person” can have an impact on this person, and this influence will change the shape of the projection on the wall.

Rudan &  Jin _Touch Designer

Reference:

Through learning some TD tutorials on Youtube and testing, we achieved the visualisation of sound in real-time. The input can be translated into flowing water, and changed based on the rhythm and strength.

Rudan’s First Test

Touch Designer_Jin

Jin’s Second Version①

Jin’s Second Version②

I used VB-audio Virtual Cable as the input and output of sounds. Therefore we can change the music at any time, just change the music in spotify or youtube, without inputing music files into Touch Designer and use switch to change. Also, it can deal with sound from microphone, environment sounds and music in place and generate animations, which we would like to project on the water ripples created by our installation.

Besides, the colors and shapes of the water ripples in TD can be changed through ramp and displace components respectively (details can be found in video and pictures①②). The size of ripples can be changed by paramater multiply in maths component.  The volume of music can also be flexible to adjust.

One obvious problem is the quality of sound, and until now I have no idea about how to improve this.

css.php

Report this page

To report inappropriate content on this page, please use the form below. Upon receiving your report, we will be in touch as per the Take Down Policy of the service.

Please note that personal data collected through this form is used and stored for the purposes of processing this report and communication with you.

If you are unable to report a concern about content via this form please contact the Service Owner.

Please enter an email address you wish to be contacted on. Please describe the unacceptable content in sufficient detail to allow us to locate it, and why you consider it to be unacceptable.
By submitting this report, you accept that it is accurate and that fraudulent or nuisance complaints may result in action by the University.

  Cancel