It’s good to see the feedback on our initial submission, and I have compiled a summary of the project’s progress so far. This blog describes our ideas, experiments, solutions and decisions in response to the tutor’s comment.
Initially, I’d like to emphasise the vital role that interactive media plays in our performance, as we strive to swiftly immerse the audience. Given the brevity of the actual performance, engaging the audience beforehand becomes invaluable as a warm-up tactic.
Reflection 1:
The primary format still requires clarification—what is the format we intend to present to the audience? Essentially, we plan to convey the content primarily through live performances by puppet shows and video projection at the main stage, complemented by captivating music and set decoration. Ideally, we aim to incorporate a blend of video and other formats displayed on the screen, such as shadow play or puppet shows. Our goal is to enhance the visual presentation and achieve an extravagant effect, thereby providing a relatively varied show to deliver an immersive experience. Moreover, we are currently investigating the potential of generating animations with the aid of Artificial Intelligence. There exist various methods and tools for production, and below are my experiments:
- Use TensorFlow’s pretrained style transfer model to transfer the style from an image to the target content image. The model inference was performed on Google Colab, with minor modification on the code to accept personalized inputs(my drawings). After several tests, I find that the style image is suppose to be photo with white spaces, which is a drawback. The modified code was generated by ChatGPT. I assume that the generated images could be put into our projection video. https://www.tensorflow.org/hub/tutorials/tf2_arbitrary_image_stylization
- Producing numerous similar images within Midjourney and assembling them into moving images or frame-by-frame animations. （AnimateDiff）
- Inputting the images to Runway Gen-2 to generate animations of up to 4 seconds. Nevertheless, challenges include the high cost and the complexity of ensuring stability and dynamism in AI-generated videos. As evident, the carps exhibit very gradual and subtle movements in the video, making it challenging to accentuate the visual impact.
- Overlaying the drawings, photographs, animations and videos captured on-site to construct a blended video by CapCut.
Reflection 2:
More importantly, it’s valuable to ponder the moral stance we aim to convey, urging resistance against our modern urges for competition, and emphasizing that fulfilment originates internally. It’s essential that the performance doesn’t simply conclude abruptly but instead offers a narrative resolution for the audience. Consider integrating elements reflecting the pervasive theme of today’s competitive lifestyle into every aspect of the performance narrative. Firstly, how to visualise the concept to the audience: how can they perceive and comprehend the theme of peer pressure within the story? One approach to tackle this is via illustrating the intense competition, highlighting the scene where carp scramble to jump. We might apply abstract ways and intense drum beats we can take to express this tension, for instance, viewing through the fish’s perspective, the splashes of water from their leaps into the world evoke a sense of rapid transformation of everything, accelerating heartbeats, and tension with a psychedelic effect. Secondly, this could involve incorporating specific sounds reminiscent of social media or email notifications into the soundscapes, complementing the narrative. We’ve now got some voice-over sound composition, infusing it with a sense of dark urgency that aligns with the overarching concept.
Reflection 3:
We’re all working hard on our own, but now it’s key to blend these elements and see how they fit in a performance, along with our physical presence. Thereby, the next step is to determine how the visuals and music of each scene work together, dedicated to a complete narrative performance –what will the audience be able to understand the narrative as being performed live? I’ll write in detail listing how the visuals and music work together for each scene.
Reflection 4:
Things to consider, listed by Andrew:
- Consider what can be performed live with real input when performing this can be particularly engaging for an audience – e.g. the voiceover for part 1 or later voiceovers.
- Lighting control – it will be important for you to still use lights but it may be necessary to use simpler lamps – you may consider building simple control dimmers/devices.
- Linking live instruments to the control of the visual story – local microphones having video implication or effect – link the live performance to the video performance/visual textures.
- The theatre set studies you highlight with soft screen projectors, backlighting and shadow puppetry are all very intriguing – it might be interesting to combine front projection with a scene or some narrative device along with backlit shadow puppetry – maybe using a localised focused light such as a torch to be able to layer different elements to your performance.
- Think very carefully about audience choreography/orchestration – how will they know to move through a space – when they need to linger, when they need to move on – this can easily be done through lighting or where audio is coming from but does need consideration to avoid needing to usher people.
- May a clear plan for each scene with what is happening – what effects, set dressing, audience location, audio is occurring and where is it coming from – this will be a working document to enable you to keep track of where you are at developing the performance.
Here are a few additional suggestions: conduct more experiments! Don’t need to invest heavily in purchases; instead, utilize waste materials and plastics to craft props.
