As the main person responsible for the production of the video, I completed the second and third parts of the video as well as the first half of the fourth part. The whole production process can be divided into several main parts:

Finding and Processing of Footage Video

Our video footage was mainly sourced from Shutterstock’s video library. After selecting the footage, I processed these videos with fine editing and effects to ensure that they would fit perfectly into our performance. For the gantry footage, I used TouchDesigner software to create a unique particle effect to enhance the visual impact.

Recording and Editing of Expression Performance

In order to express the changing emotions of the carp at different stages of the dance, I have recorded several special expression performances. These expression videos will be played in the background of the dance, echoing the movements of the dancers. After recording, I carefully edited these videos and enhanced their artistic effects through post-processing to ensure that they could truly and vividly convey the inner world of the carp.

Animating with TouchDesigner

There are several key images in the video that required special production techniques. For example, the tornado, the waves and the fish swimming head-on were all particle animated using TouchDesigner software.

 Tornad

For the tornado animation, I designed a system that could be manipulated based on mouse position. By manipulating the mouse in the software, I recorded the ideal tornado motion trajectory to make it look more realistic and dynamic.

Wave

When creating the wave effect, I initially tried the sound control method, hoping that the waves would rise and fall with the rhythm of the music. However, after actual testing, the result was not satisfactory. After repeated tests, I finally chose to use constant value to simulate the movement of the waves, which can better control the overall effect of the picture.

Fish swimming head-on

The image of the fish swimming head-on is to create a feeling of suffocation and oppression. By adjusting the density, speed and direction of the particles, I succeeded in creating the effect of a large number of fish swimming head-on, which makes the audience feel as if they are in the water, feeling the helplessness and struggle with the carp.

heartbeat and the dragon gate

For parts such as the heartbeat and the dragon gate, I used TouchDesigner to further process the footage video by adjusting the colours, contrast, particle effects, etc. to make the image more artistic and abstract to match the overall atmosphere of the performance.

Video production with the help of AI technology

For the part of the carp going up to the sky, we wanted to create a surreal and dreamy effect. For this, I used Runway, an AI-assisted video production tool. With Runway, we can transform still images into moving videos and add various creative effects. Using this technique, I managed to bring to life the scene of a carp going up to the sky, adding a touch of fantasy to the whole performance.

Apr 10, 2024

The first rehearsal is a crucial stage in our stage performance project, which gives us the opportunity to test the venue set-up and equipment commissioning in a real-world environment.

During the process, we made several attempts to test the projection effect of the projector, the induction sensitivity of the light strips, and the lighting effect of the fish lights. We also explored different setup options, including wool wraps, light decorations, and foam paper effects.

Equipment Commissioning

The projector was our main visual device, and we adjusted the projection angle and brightness during the rehearsal to ensure the best projection effect. The sensing sensitivity of the light strips, on the other hand, has a crucial impact on the synchronisation between the music and the light effects, but in the test, the strips had difficulty in accurately capturing the sound changes in the scene.

Venue Setup

In terms of venue setup, we first tried the wool winding solution. However, the test results did not meet our expectations. The combination of wool and lights was average and did not create the visual effect we hoped for. We also tried light decoration, but again, the desired effect was not achieved.

Finally, we tried the foam paper effect. This effect was unexpected, it added a hazy feel to the venue and we decided to keep this design. However, we found that the foam paper was easy to stick diagonally and we plan to improve this before the official performance.

Apr 1, 2024

Firstly, I picked up a 5 metre roll of WS2812B RGB LED Pixel Strip. This strip features individual colour control for each LED pixel, making it ideal for flexible and versatile lighting effects. Next, I chose the Arduino software as the control platform to enable precise control of the lights for live music.

Initially, I used the LM393 sound sensor to capture the sound signal of the live music, and then tried to write code to convert the sound signal into lighting effects. However, after many attempts and adjustments to the code, the results were still not satisfactory.

After a series of searching and learning, I decided to try to use MAX9814 sound sensor instead. MAX9814 sensor has higher sensitivity and performance to capture the music signal more accurately, which improves the accuracy and response speed of the lighting effect. And the code logic was realigned. In the original code, the update of the lighting effect is executed based on a fixed delay, which may result in being out of sync with the music tempo. In the improved code, an interval-based update mechanism is introduced, using the millis() function to achieve timed updates. This allows for more precise control of the update frequency of the lighting effects, allowing them to be better synchronised with the music tempo.

By increasing the sensitivity of the sensor and optimising the structure of the code, I managed to improve the capture of the music signal and reduce the latency of the code to make the lighting effect smoother.

Final code:

#include <FastLED.h>

#define LED_PIN 5

#define NUM_LEDS 300

 

CRGB leds[NUM_LEDS];

int soundsensor = A0;

j

void setup() {

delay(2000);

Serial.begin(9600);

FastLED.addLeds<WS2812B, LED_PIN, GRB>(leds, NUM_LEDS);

FastLED.setBrightness(50);

}

 

void loop() {

int soundValue = analogRead(soundsensor);

 

Serial.print(“Sound level: “);

Serial.println(soundValue);

 

// int redValue = 0;

int blueValue = 0;

 

if (soundValue > 70) {

redValue = map(soundValue, 80, 1023, 0, 255);

fill_solid(leds, NUM_LEDS, CRGB(redValue, 0, redValue));

} else if (soundValue > 40) {

blueValue = map(soundValue, 50, 80, 0, 255);

fill_solid(leds, NUM_LEDS, CRGB(0, 0, blueValue));

} else {

fill_solid(leds, NUM_LEDS, CRGB::Black);

}

 

FastLED.show();

delay(20);

}

Next, I secured the light strips to foam tape to create a water ripple shape and positioned them in the area between the stage and the audience.

Apr 1, 2024

 

In our stage performance project, we originally planned to place an ice screen in front of the main screen to project the gantry image and the expression performance to express the emotions of the little fish. Actors could travel in front of and behind the ice screen during the performance, creating a before and after scene.

However, during the testing phase, we encountered some challenges:

1. The superposition of the front and back stage structures did not meet our expectations in terms of visual effect. There was a lack of harmonious integration between them, which in turn appeared to be abrupt and distracted the audience’s attention.
2. The size ratio of the ice screen is 1:2, which is quite different from the ratio of the projected image of the projector, causing difficulties in video production.
3. The light source of the small projector would penetrate the translucent ice screen and shine on the main screen, which affected the picture effect of the main screen.

For these reasons, we decided to move the ice screen to the right side of the stage and adjust the content strategy. We dropped the originally planned gantry screen and small fish emoji performance and instead created an abstract screen that could interact with the performers in real time. I created a TouchDesigner file to achieve this, and chose a blue colour palette that echoed the theme of the stage to simulate the flowing aesthetic of liquid water. Driven by the live sound, the dots of light in the image would flicker and flicker in response to the intensity of the sound, resembling the light jumping off the surface of the water.

At first I tried to use the audioAnalysis block to capture the low, mid and high frequencies, spectrum, tempo and volume of the live sound, in order to generate a varied and layered visual effect. However, after a series of tests, I found that this approach did not work as well as I had hoped when combined with live sound. So I turned to the switch component. Although switch is not as sophisticated as audioAnalysis in terms of functionality, it shows a much better and more stable performance in the field.

Final Demo

Right side screen live effect

Apr 1, 2024

We explored a variety of materials and crafts to create a wide range of props for this theatre performance project.

Four Little Fish

Upper stage small fish

The overall design of two of the small fish is similar to that of the big fish, but we used material packages to make the process much simpler to save unnecessary work. These two small fish have light strips wrapped around their interiors, enabling them to emit a faint glow on stage, enhancing the visual effect. They will be on stage in three of the acts.

Lantern Fish

The other two small fish are decorated on top of the lanterns to mimic the form of a fish. The red fish is controlled by Hongpei Cao and performs in real time, with its performance projected on the left screen. The pink fish is used as a live decoration.

Decorative Goldfish

We also made decorative goldfish. We used glossy cardboard and drew goldfish patterns on it. Although the original plan was to hang these goldfish on the stage, we finally decided to place them on the wall by the passageway for safety reasons.

Balloons

We hung different coloured balloons at the entrance, including transparent, light blue and dark blue. The transparent balloons were also filled with tricolour wool inside. When viewers wish to enter, their faces are blocked by the balloons, which creates a suffocating, crowded and oppressive feeling, symbolising the crowded and oppressive existence of carp in the lake.

Foam Tape on the Floor

We noticed during rehearsals that the audience would have trouble determining where they wanted to stand, so we decided to place a foam strip on the floor to differentiate between the stage and the auditorium. The foam paper will be painted with a blue water pattern, suggesting that the story takes place in water.

Mar 27, 2024

初步设想

我希望我们的舞台不仅仅是艺术追求，更是对社会现象的深刻反思。我的想法触及当前环境危机的核心和人类情感记忆的复杂性。

海洋污染与人类责任

我的第一个项目的灵感来自最近引起广泛关注的日本核废水事件。此次事件不仅反映了海洋污染的严重状况，也反映了人类活动对环境的长期影响。故事讲述了两条小鱼在海里快乐地游荡的故事。突然有一天，身后传来黑色的污水，小鱼开始了艰难的逃亡之旅。途中，他们失去了家人，回忆起过去的快乐时光。然而，最终，黑色的海水却无情的吞噬了他们。这不仅是对生态危机的警告，也是对人类行为后果的深刻反思。通过小鱼的视角，我们可以感受到海洋生物所面临的绝望和无助，进而唤起人们对环境保护的意识和责任。

对于海洋主题，我对视觉效果也有一些想法 

 

黑暗童话

Jan 31, 2024