The first rehearsal is a crucial stage in our stage performance project, which gives us the opportunity to test the venue set-up and equipment commissioning in a real-world environment.

During the process, we made several attempts to test the projection effect of the projector, the induction sensitivity of the light strips, and the lighting effect of the fish lights. We also explored different setup options, including wool wraps, light decorations, and foam paper effects.

Equipment Commissioning

The projector was our main visual device, and we adjusted the projection angle and brightness during the rehearsal to ensure the best projection effect. The sensing sensitivity of the light strips, on the other hand, has a crucial impact on the synchronisation between the music and the light effects, but in the test, the strips had difficulty in accurately capturing the sound changes in the scene.

Venue Setup

In terms of venue setup, we first tried the wool winding solution. However, the test results did not meet our expectations. The combination of wool and lights was average and did not create the visual effect we hoped for. We also tried light decoration, but again, the desired effect was not achieved.

Finally, we tried the foam paper effect. This effect was unexpected, it added a hazy feel to the venue and we decided to keep this design. However, we found that the foam paper was easy to stick diagonally and we plan to improve this before the official performance.

Apr 1, 2024

After a few collaborative rehearsals, I laid down a detailed performance schedule – down to the minute, who was on stage, what they were going to do, and so on. It was my job to take control during the show, cue the actors on stage and what they were performing, and make sure the show flowed smoothly.

To control the situation with precision,  as the visual director, I tried various forms of documenting and articulating the performance schedule to ensure that each member understood their task.

Written proposal: DIRECTING

Video narration: this is based on projected video with subtitles to cue the actors to where they are. Certainly, this won’t be on the final show, but for rehearsals, it’s a good way to get everyone to understand the action.

 

Morover, I designed clear lines of movement and waiting position, which to avoide chaotic collision when the actors need to intersperse their performances—

• The carp only initially entered from the east, all the rest of the entrances and exits were behind the ice screen on the west side.
• The waiting position for the two small fishes is always on the west side of the screen.
• The waiting position for the dancers is always on the west side of the screen.

 

On top of that, I personally demonstrated all the characters twice so that the actors could grasp the techniques. For example, how the little fish trembles when struck by lightning, how the carp’s body changes when it ascends into the air, and so on. I devised and agreed with the group on hand signals such as going up, retreat, running left, running right, jumping, etc. that would remind the actors to act during rehearsals. We originally wanted to talk through headphones or walkie-talkies, but this might affect their judgement of the live sound effects, so we finally decided to dispatch with hand signals.

Apr 1, 2024

Firstly, I picked up a 5 metre roll of WS2812B RGB LED Pixel Strip. This strip features individual colour control for each LED pixel, making it ideal for flexible and versatile lighting effects. Next, I chose the Arduino software as the control platform to enable precise control of the lights for live music.

Initially, I used the LM393 sound sensor to capture the sound signal of the live music, and then tried to write code to convert the sound signal into lighting effects. However, after many attempts and adjustments to the code, the results were still not satisfactory.

After a series of searching and learning, I decided to try to use MAX9814 sound sensor instead. MAX9814 sensor has higher sensitivity and performance to capture the music signal more accurately, which improves the accuracy and response speed of the lighting effect. And the code logic was realigned. In the original code, the update of the lighting effect is executed based on a fixed delay, which may result in being out of sync with the music tempo. In the improved code, an interval-based update mechanism is introduced, using the millis() function to achieve timed updates. This allows for more precise control of the update frequency of the lighting effects, allowing them to be better synchronised with the music tempo.

By increasing the sensitivity of the sensor and optimising the structure of the code, I managed to improve the capture of the music signal and reduce the latency of the code to make the lighting effect smoother.

Final code:

#include <FastLED.h>

#define LED_PIN 5

#define NUM_LEDS 300

 

CRGB leds[NUM_LEDS];

int soundsensor = A0;

j

void setup() {

delay(2000);

Serial.begin(9600);

FastLED.addLeds<WS2812B, LED_PIN, GRB>(leds, NUM_LEDS);

FastLED.setBrightness(50);

}

 

void loop() {

int soundValue = analogRead(soundsensor);

 

Serial.print(“Sound level: “);

Serial.println(soundValue);

 

// int redValue = 0;

int blueValue = 0;

 

if (soundValue > 70) {

redValue = map(soundValue, 80, 1023, 0, 255);

fill_solid(leds, NUM_LEDS, CRGB(redValue, 0, redValue));

} else if (soundValue > 40) {

blueValue = map(soundValue, 50, 80, 0, 255);

fill_solid(leds, NUM_LEDS, CRGB(0, 0, blueValue));

} else {

fill_solid(leds, NUM_LEDS, CRGB::Black);

}

 

FastLED.show();

delay(20);

}

Next, I secured the light strips to foam tape to create a water ripple shape and positioned them in the area between the stage and the audience.

Apr 1, 2024

 

In our stage performance project, we originally planned to place an ice screen in front of the main screen to project the gantry image and the expression performance to express the emotions of the little fish. Actors could travel in front of and behind the ice screen during the performance, creating a before and after scene.

However, during the testing phase, we encountered some challenges:

1. The superposition of the front and back stage structures did not meet our expectations in terms of visual effect. There was a lack of harmonious integration between them, which in turn appeared to be abrupt and distracted the audience’s attention.
2. The size ratio of the ice screen is 1:2, which is quite different from the ratio of the projected image of the projector, causing difficulties in video production.
3. The light source of the small projector would penetrate the translucent ice screen and shine on the main screen, which affected the picture effect of the main screen.

For these reasons, we decided to move the ice screen to the right side of the stage and adjust the content strategy. We dropped the originally planned gantry screen and small fish emoji performance and instead created an abstract screen that could interact with the performers in real time. I created a TouchDesigner file to achieve this, and chose a blue colour palette that echoed the theme of the stage to simulate the flowing aesthetic of liquid water. Driven by the live sound, the dots of light in the image would flicker and flicker in response to the intensity of the sound, resembling the light jumping off the surface of the water.

At first I tried to use the audioAnalysis block to capture the low, mid and high frequencies, spectrum, tempo and volume of the live sound, in order to generate a varied and layered visual effect. However, after a series of tests, I found that this approach did not work as well as I had hoped when combined with live sound. So I turned to the switch component. Although switch is not as sophisticated as audioAnalysis in terms of functionality, it shows a much better and more stable performance in the field.

Final Demo

Right side screen live effect

Apr 1, 2024

The music in this section is derived from previously uploaded original music composition demo, but with some percussion samples added to the original.

This part of the original plan was considered to set up some digital media content: the sensors were set up on the drums and played live, so as to influence the video footage through the live drum playing. However, in practice, we found that the sensitivity of the sensor was too difficult to manipulate in such a sound environment, and was prone to errors during live performance, so we changed this stage to direct percussion.

The percussion samples come from a Chinese national instrument, the demi-drum, which simultaneously produces the sound of a drum being struck, the sound of a bell, and the sound of metal rubbing against metal. In this piece of music, some of these drum samples are added and played live, using the instrumentalist’s walking route as a clue to lead the audience from the corridor into the formal performance venue.

To furthermore, percussion and vocal samples are also used in this piece. The vocal samples originate from the north of China, and this special singing style is called hu mai, which is a special method of singing to achieve the effect of singing in harmony with one person, with a unique ethnic characteristic.

Mar 31, 2024

In part 2, in order to state the background of the story (fish live in depressing and crowded waters), the color of the music in this part turns low and depressing, and the 0 to 43 seconds need to give way to sound space for the lines, which are mainly produced as background music. 43 seconds later, the dance joins in, and there is a strong twist in the visual effect, in the case of the original piano as the base, the more intense percussion and string groups were also added.

YouTube link: https://youtu.be/p-FCcYl4fQI?si=8i1Qozo0kEq6yYZo

In the third part, the music is accompanied by the sound effects of the water flow, which is also composed as an aural effect with fluidity, consisting mainly of delayed electronic pads. At the same time, the percussion in the second half of this piece of music is joined by a processed electronic sound similar to poor contact, in order to articulate the next part of the fifth part of the segmentation of different music.In this section, the music is less melodic than in the previous sections, partly because of the need to give way to the lines, and partly to match the sound effects and balance the frequency range.

Mar 30, 2024

The first rehearsal was postponed due to the length of time it took to arrive at the production for our show and the complexity of setting up a 5.1 sound system at the same time. During this rehearsal, we identified a number of issues for improvement.

Firstly, the actors’ movements need to be further explored and improved. During the rehearsal, we found that the carp characters were not very expressive and it was difficult to show rich emotions, and the use of the carp characters also limited the actors’ movements – the actors could only lift the carp characters to do some simple movements. So from the audience’s point of view, the whole plot and story line of the performance, including the emotions of the characters, were not expressed very clearly.

Secondly, after a group discussion, we agreed that we needed to add some more lines and dialogues. We found that the existing acting, music and sound effects were not sufficient to adequately convey a clear plot and storyline.

In addition, we found the realisation of the original dance section to be more difficult than we had imagined. Our original plan was to have members of our group perform the dance, but after rehearsals we found that the non-professional dance performances were not expressive enough, so I contacted my friend who is a dance major and communicated with her about the overall timeline, story and content. Expressed emotions and invited her to join us for the performance.

Also, after completing the setup, we realised that the venue setup (sound system, decorations, projector and screen setup, etc.) was taking much longer than planned, and we had to find a way to shorten this time while arriving and setting up the venue a little earlier than planned on the day of the official performance.

Finally, in the music section, I think some adjustments need to be made based on dance time adaptations and line additions. On the one hand, some of the music needs to give way to the frequency of the lines and dialogue; on the other hand, the music should have a greater sense of rhythm and passage when joining the dance to provide better conditions for the dancers to play.

And the singing part of the rehearsal.

Mar 29, 2024

Mar 28, 2024

We explored a variety of materials and crafts to create a wide range of props for this theatre performance project.

Four Little Fish

Upper stage small fish

The overall design of two of the small fish is similar to that of the big fish, but we used material packages to make the process much simpler to save unnecessary work. These two small fish have light strips wrapped around their interiors, enabling them to emit a faint glow on stage, enhancing the visual effect. They will be on stage in three of the acts.

Lantern Fish

The other two small fish are decorated on top of the lanterns to mimic the form of a fish. The red fish is controlled by Hongpei Cao and performs in real time, with its performance projected on the left screen. The pink fish is used as a live decoration.

Decorative Goldfish

We also made decorative goldfish. We used glossy cardboard and drew goldfish patterns on it. Although the original plan was to hang these goldfish on the stage, we finally decided to place them on the wall by the passageway for safety reasons.

Balloons

We hung different coloured balloons at the entrance, including transparent, light blue and dark blue. The transparent balloons were also filled with tricolour wool inside. When viewers wish to enter, their faces are blocked by the balloons, which creates a suffocating, crowded and oppressive feeling, symbolising the crowded and oppressive existence of carp in the lake.

Foam Tape on the Floor

We noticed during rehearsals that the audience would have trouble determining where they wanted to stand, so we decided to place a foam strip on the floor to differentiate between the stage and the auditorium. The foam paper will be painted with a blue water pattern, suggesting that the story takes place in water.

Mar 27, 2024

Research

I attended the Edinburgh Blues and Jazz concert and found their set interesting – the projected video responded to the rhythm of the music so that the audience could simultaneously feel a stronger resonance on the audition. They take on abstract and realistic usages for different music and scenarios.

1. https://drive.google.com/file/d/1PD_YsehuFe_06th2SG4sGAz_gSwsjAM4/view?usp=sharing
2. https://drive.google.com/file/d/1mgz00vzciFigeMMB6k7DlakCXpEtsSZn/view?usp=sharing

I learned to create animations that change according to the music, and decided to merge them into the projection video.
1. https://drive.google.com/file/d/17co5xMl2L9zar8gS288Y6A_Am6pRNyUU/view?usp=drive_link
2. https://drive.google.com/file/d/1uO2745sN5jUQqO-M_TaDL6bcFji9Yx3L/view?usp=sharing

 

Design of audio-visual coordination

To direct the audience’s attention to a specific screen, we ingeniously used 5.1 surround sound along with strategic changes in the front and back screen spaces. In this way, the audience is able to naturally turn in the desired direction based on the visual-sound interaction.

 

Light and shadow

The shadow play in the opening credits aims to create mystery and intrigue. At the end of the movie, the shadow play mirrors the opening, implying the carp has returned to its cage.

Mar 26, 2024