• • Donger LIU:

1. 1. Visual director: overall aesthetic
   2. Illustration/animation design
   3. Handcrafts: set dressing, props making
   4. Adaption of scripts
   5. Portfolio producer, with study annotations
   6. Managing resources allocated to visual design

• • Honepei Cao

1. 1. Sound effects design
   2. Max interactive audio installation design
   3. Max particle design
   4. Lighting control
   5. Surround sound environment design
   6. Sound recording and editing

• • Qiyan Zhang:

• • Xianni Sui:

1. 1. Music Composition
   2. Sound Recording
   3. Music Instruments Performance
   4. Background Research
   5. Surround Music Effect design

• • Fangyi Qu:

1. 1. Sound effects design
   2. Touch designer design
   3. Surround sound environment design
   4. Sound recording and editing
   5. Max and Touch Designer interactive
   6. Risk assessment

Rough Script (15-20 minutes)

Set Dressing Mood References

Making screens out of fabric

Ballons imitate crouding fish / Fish lantern exhibition at Shanghai

Theatre Study

It is a theatre with a screen made of transparent strip curtains, named Hamlet. Directed by Thomas Ostermeier.

We would like to apply this idea in our performance to introduce audiences to the more significant stage site behind, letting them find the joy of exploration in this.

The installation Christian Boltanski by Marcos Rabello, shows the flickering light cast onto the wall and the shadows of little figures. We intend to cast audiences’ shadows in this way, transform them from spectatorship to being part of the performance and enhance participation. However, we need to address and experiment with the issue of audience members misunderstanding that they are blocking the projection and hiding from the lights. Our goal is to find effective ways to communicate to the audience that they are invited to participate.

Christian Boltanski

Experiment

A test of the projector inspired by Christian’s work

Interactive Installation Research

Explore the possibility of creating a soft screen by projector and curtain for audience members to walk through.

Use different coloured lights to highlight the main characters. Or consider, shadow show can also be a form.

Site Inspection

We aimed for an immersive stage experience, divided into three sections: the entry tunnel (orange), performance area (blue), and viewing area (green).

The entrance allows audiences to pass through a narrow, decorated passageway resembling the ocean environment of carp. The narrow space is decorated with balloons and coral to simulate the environment of carp in the ocean. We aim to let the audience understand the survival of the carp in the water through this crowded environment.

Upon entering the viewing area, projections in the performance area and on the sides will enhance immersion. Yet, details regarding projection equipment usage and room setup are still pending.

We propose to make an additional foreground curtain on the left side of the central performance area. This versatile curtain serves as both a projection surface and an entry/exit point for actors, enhancing stage hierarchy and facilitating actor movement.

Next, we will visit the designated rooms for field measurements to make necessary adjustments and refine the design for accuracy based on the room’s dimensions.

The Modified Design

We made modifications to the setup based on the data from the field measurements (right image).

As you can see, the room will be divided into a small square film theatre in the centre, with a ring of encircling passages around it. Red arrows show the viewing route, which will then be guided by props.

The image below is the previous set dressing idea, which references and illustrations are provided to illustrate the prototype.

Concept Illustration

To illustrate the conceptual design of our set, I have drawn a schematic storyboard to follow an audience member through the experience of our performance from a third-person perspective.

1.1) There will also be an interactive screen in the passway, adding engagement.

1) The entrance passway is decorated with balloons to simulate the fish crowding.

2) Fish lamps hanging overhead, introduce the context of their living conditions.

3) Follow the projection with crowding moving forward, walking through the curtain to see the next scene behind.

4) Follow the projection with crowding moving forward, walking through the curtain to see the next scene behind.

5) Follow the projection with crowding moving forward, walking through the curtain to see the next scene behind.

6) All lights go out, leaving only bottom lighting behind the audience. Their shadows merge with the projected aquarium.

Projected Video Demo

The projecton background mainly explains the environment in which the story develops. The projecton is mainly divided into three scenes: the river environment that creates the life of the carp, the rough river surface when the carp leaps the Dragon Gate, and the carp leaping through the empty door to enter the Heavenly Court.

As a traditonal Chinese story, the design of the background also incorporates Chinese style, hoping to reflect the cultural background of the story.

Projection Scene Demo 1, Rough river, painted with colour blocks, generated with AI

Projection Scene Demo 2, Reach the Heavenly Palace, painted with colour blocks, generated with AI

Projection Effect Test

1. 1. Chromatic Aberration Problem

During our testing of the place andprojector, we discovered that the colors did not match as anticipated, so that the mage did not appear as expected.

1. 2. Size adaptability between Projection andImage.

After many attempts, we found that the range and distance of projection equipment we currently use do not match the images in our works.

Response:

We would investigate the next steps, practice the application of the projector, adjust the scale of the image and be flexible in the choice of projection equipment according to practice.

Technical Details

Touch Designer for Interactve Dialogue and Background

Touch Designer Image Partcle

Scene One: crowding living conditions.

Particle effect: using particle technology to simulate the crowding in the river.

With Touch Designer, we could create streams of particles that look like fishes, which allows the audience to control the fish swimming directions by waving their arms.

Scene Three: toward the illusory dragon gate.

Particle effect: the particle gantry may change suddenly, releasing light particles that appear to attract but push the carp toward the trap.

Visual cue: as the carp get captured, the particles on stage shift from an orderly flow to an eruption of chaos, symbolising betrayal and pain.

Scene Five: fall into the aquarium

Particle effect: in the end, the particles in the scene can gradually disappear until the stage is completely darkened, leaving the audience to ponder.

Touch Designer Camera Hand Recogniton Partcle Interacton

Scene Five: fall into the aquarium

Partcle farewell: with the carp’s “last breath”, the audience can gently wave hands to make the partcle rise slowly, symbolising the departure of the carp’s soul and the end of the story.

Touch Designer Image Partcle

Scene 2: A large touch screen that acts as an interactve “canvas” where viewers can touch and “paint” on the screen, producing music and visuals with each of their touches.

1. Sound trigger: Max/MSP captures the sound produced by touch and processes and synthesizes sound effects according to different touch positons and intensifies. Each touch corresponds to a specific sound effect

2. Visual feedback: TouchDesigner receives audio signals generated by Max/MSP, and then generates real-tme visual effects based on audio propertes (such as pitch, volume, rhythm). For example, a bass might produce a blue ripple, and a treble might produce a red spark.

3. Audience partcipaton: The installaton will allow multple audience members to interact simultaneously to create a dynamic musical and visual artwork.

Technical implementaton

1. Touch input: Use the touch screen to capture the movement and touch of the audience.

2. Max/MSP: Analyze touch data and convert it into musical elements. Use audio synthesis and processing to create music that responds to touch.

3. Send Max/MSP music elements to TouchDesigner via OSC or MIDI.

4.TouchDesigner: Receive music data and create corresponding visual effects. Visual effects can be dynamic geometric shapes that map directly to musical features, or they can be more abstract visual representatons such as colour gradients or light and shadow flows.

5. Visual output: The visual effect is projected back onto the interactve canvas, synchronizing with the audience’s touch to form a complete audio-visual experience.

Max8 for Sound and Live Particles

General Description

In our project, the application of Max8 is primarily focused on audio interaction in Scene 1 and in creating the atmosphere for subsequent horror or mysterious scenes.

For the audio interaction, we may use Touch Designer in conjunction with ProTools for sound design to accomplish audio triggering. If it’s technically infeasible to achieve stable interaction between Max MIDI output and Touch Designer, we will resort to using only Touch Designer for image output. This output will be linked to Max8 via the same touch input to simultaneously trigger audio playback in Max.

This approach carries the risk of audio-video desynchronization, so our priority is to use a single software for production

Interective Sound Installation

For this part , we’ve created a demo of interactive sound effects using ProTools’ 5.1 surround sound format. This demo ranges from normal whispers with a slight reverb to distorted, heavily reverberated sounds resembling a dragon’s rumble, with added surround sound effects.

Given the space constraints in the Scene 1 corridor where the actual setup will take place, it might not be feasible to use surround sound speakers. To address this, we can render the surround sound content in binaural format, which is then outputted to headphones.

This approach allows for an immersive audio experience using a headphone amplifier, supporting multiple audience members simultaneously. This method ensures that the spatial quality of the sound is maintained, providing a realistic and engaging auditory experience that complements the visual and interactive elements of the scene.

https://drive.google.com/file/d/1eaQXOGu59cZgbWMIUCUkM2u7VTatm0as/view?usp=share_link (5.1 dialogue sound design

Possible lines:

“Beyond the gate, freedom calls.”

“Leap into your destiny, find your strength.”

“Embrace the unknown, for it holds your liberation.”

“In the leap, lies the path to transformation.”

“Dare the impossible, become the unimaginable.”

“Seek the gate, embrace your true form.”

“The gate beckons, promising a new dawn.”

“Cross the threshold, claim your power.”

“In the leap, your chains break.”

“The gate is the key, unlock your potential.”

Particle Animation

Another aspect is the video particle feature in Max8. Due to the complexity of implementing particle effects in Max8, with fewer plugins and preset options available, it requires inputting advanced functions for real-time calculation of particle positions.

After preliminary attempts, we managed to achieve the demonstrated particle movement effects and tested them using a portable projector. The actual effects look very impressive in a completely dark environment. We can use the reverse motion of particles, coupled with sound effects, to create a dispersing effect, simulating the inner turmoil of the protagonist Yue or creating an atmosphere. We are also exploring the use of OpenProcessing and other software, aiming to leverage a vast array of presets and public cases to create more visually appealing scenes.

ADJ MyDMX Controller for Lighting Control

The technology for controlling lighting is somewhat limited. After reviewing the lighting equipment available for rental on Bookit, it appears that the options for small-scale performances, especially in terms of lighting controllers, are quite restricted.

There is only one portable ADJ control box and a single console available. However, the console is frequently occupied for extended periods by various events. This situation limits the practical use to just two large LED lights that can be operated simultaneously. These two lights are mainly used for creating special effects, such as flashing lights to enhance the atmosphere during dramatic moments like the ‘Crossing the Dragon Gate’, or in Scene 5’s aquarium setting, where they project the audience’s shadows onto the screen.

Protools & Live for Immersive Sound Design

In the current design, Protools is used to create all sound effects and simulate the desired interactive effects. If some of the audio interactions mentioned earlier cannot be achieved, we can substitute them with live control through Protools and the mixing console as a backup plan. Considering that each act of the performance might have a fixed duration, pre-produced surround sound tracks can significantly reduce the risk of performance issues and allow for multiple tests to achieve the best effect. If we opt for a format with variable durations, we could consider using Ableton Live for looping the ambient sound tracks.

Protools Session Dialogue Design

Sound Environment Demo Audio Link: https://drive.google.com/file/d/1fp9mix5ti7y1oirtB_dtwCjSIWte-D0t/view?usp=share_link

Original music composition demo & Cultural Background

Link of demo:

This piece is an original theme music composition for our project, it is related to the background, the development of the storyline, and the overall thematic significance of the story. It is suggested that, the Chinese have long used images to convey auspicious wishes, such as plants, flowers, animals, birds and objects. (Laing, 2017). In this situation, fish express hopes for wealth because the pronunciation of the character for “fish” is identical to “abundance.”This is based on the tonal nature of the Chinese spoken language, one sound could have different meanings. In this context, carp were especially favored, because of their jumping ability, they were always associated with the idea of leaping through the Dragon Gate (an ancient Chinese myth) which symbolizes the attainment of success.

To furthermore, this original demo maintains plenty of Chinese traditional elements, for example Chinese pentatonic scales, as well as a part of Chinese traditional music instruments. The entire composition is structured into three distinct parts:

To begin with, the first section delineates the environment, spotlighting the protagonist (the carp) navigating its existence within the polluted waters, some Chinese traditional instruments like Instruments‘XUN’, evoked a somber and murky ambiance.

Additionally, the second segment delves into the character’s actions, emphasizing its leap over the dragon gate and the ensuing series of endeavors.

Next, the third part crescendos into the climax of the piece.Instruments such as the Suona, Pipa, and Bamboo Flute build upon the atmosphere layer by layer, culminating in a climactic finale. The unique technique of the Suona unveils the resolution of the story.

Reference list:

Laing, Ellen Johnston. “Carp and Goldfish as Auspicious Symbols and Their Representation in Chinese Popular Prints.” Arts Asiatiques 72 (2017): 97–109. http://www.jstor.org/stable/44656691.

Risk Assessment

1. 1. Insufficient budget: materials for the set dressing, such as iron frame and curtain to be purchased, which leads to a high cost. The soluton is to adjust the scope of the project without compromising the core objectves of the project, and examine the project budget to evaluate the unnecessary expenses that can be cut or replaced with something else at a lower price. This may include reducing certain features, simplifying the design, or changing the way the project is delivered.

1. 2. Shortage of equipment: we have to book the required equipment two weeks in advance before our performance, (Bookit could be booked up to two weeks in advance). Moreover, we should consider some alternatve equipment as plan B.

1. 3. Technical challenges: seek advice and help from relevant professionals when computer system problems lead to program errors. the connecton between devices is a big difficulty because of the need to use sensors and touchable screens to interact. For wirelessly connected devices such as ZIG SIM and touch designer, check and test the network environment in advance. For wired devices such as lights and computers, prepare spare cables in advance to prevent problems with the wiring.

1. 4. Insufficient project human resources: due to the shortage of site resources, we have to arrange more devices, arriving at the site in advance is necessary. Prioritse tasks (ensure that the most important tasks are assigned to a single person to complete, and one person can take charge of two tasks for simple or minor tasks). Find friends who are willing to help and invite them to help set up the scene once the tme is not enough.

Further Expectaton

To begin with, try out and solve problems with the practcality of all technologies, then adjust and improve them, including equipment, sound effects, music, lightng control and image display.

Secondly,to enhance project completon, team members created props, sound effects, lightng, and music, all while staying in touch, sharing ideas, and coordinatng the tmelines to ensure each component is finished on schedule.

Finally,stay vigilant for any project issues as they arise, and address them promptly to enhance workflow efficiency.

Play Script (15-20 minutes)

Scene One: The Polluted River

• Background: Dim, murky waters with dead fish floating.
• Action: The carp struggles amidst the pollution, hearing of the Dragon Gate legend.
• Dialogue (Possibly shown by screen): Expresses a yearning to escape and the despair felt.
• Location: Alison House Corridor
• Installation: TV with touch screen or sensor, fish models, 2 speakers

Scene Two: The Harsh Journey

• Background: Dangerous, uncertain river paths.
• Action: the carp faces various challenges and witnesses the tragic fate of companions.
• Dialogue (Possibly shown on screen): Internal struggle and fear.
• Location: Alison House Atrium
• Installation: Stage setting, 5 speakers

Scene Three: The Illusory Dragon Gate

• Background: Imposing, ominous Dragon Gate, concealing traps.
• Action: the carp attempts to leap over, only to be captured.
• Dialogue (Possibly shown on screen): Outburst of pain and feelings of betrayal.
• Location: Alison House Atrium
• Installation: Stage setting, 5 speakers

Scene Four: Aquarium Imprisonment

• Background: Bright, contrastingly cheerful aquarium, stark against carp’s turmoil.
• Action: the carp becomes an exhibit, and sinks into despair.
• Dialogue (Possibly shown on screen): Reflections on freedom and truth.
• Location: Alison House Atrium
• Installation: Stage setting, 5 speakers

Scene Five: The Final Awakening and Departure

• Background: Dimly lit aquarium display.
• Action: the carp closes its eyes, slowly sinking to the bottom.
• Dialogue (Possibly shown on screen): Inner surrender and acceptance of reality.
• Location: Alison House Atrium
• Installation: Stage setting, 5 speakers

• Portraying marine pollution in an abstract way（plastic/nuclear waste/overfishing）
• Portraying marine pollution through the perspective of ocean creatures, e.g., mainly focusing on their living conditions, with the above points.

It could be a work to point out the destructive behaviour of humans towards nature. Due to the epidemic, war, nuclear radiation and other reasons, the ocean creatures will begin to mutate and chaos.

WHY: I’ve chosen this topic in response to reports of Japan’s ongoing discharge of nuclear wastewater into the ocean. The contamination spreading through seawater threatens global oceans and people worldwide. However, many still need to be made aware of the issue. The performance aims to raise awareness and shed light on this critical issue. It is an engaging way to portray marine pollution and highlight the significance of environmental protection. Besides, I am passionate about marine and fish crafts as making stage props installations such as fish lanterns.Second-a-Day

HOW: in terms of the visual effects, my very first reference to show is an illustration I’ve done before, which described the horrible world after that chaos. It is beneficial for other members to visualize concepts and discern potential avenues for further exploration.

Here are more images as the presenting formats found by Qiyan:

Andrew suggested we think about the narrative structure behind this theme, leading me to go further on research; I found the antiwar film <Most Shocking Second a Day> by an advertising company, Don’t Panic, and a community organisation, Save the Children. The film is based on the true story of a Syrian asylum seeker, contrasting her once-happy life with the upheaval she experienced after the war. This contrasting technique is heartwarming and brings the audience closer to the experience.

Overall, this is the general orientation I’ve got in the initial stage, and we’ll have a meeting to decide our final topic after. For the next step, I will set up a study plan with a Gantt Chart to ensure our project is feasible in time, allocate jobs, and start on further research of storyboard/experiments…etc.

关于勇气

我的第三个想法反映了从童年到成年的个人成长和内部挑战。小时候，父母的严格限制让我变得谨小慎微。作为一个成年人，尽管获得了更多的自由，但由于过去的限制，我发现自己畏缩并不敢冒险。我意识到真正阻碍我前进的并不是外界，而是我自己的恐惧和缺乏勇气。

The Ideas on Main Themes

Scene of Nuclear Waste

After watching the television series ‘Chernobyl’, my understanding of nuclear contamination has deepened significantly. However, there remains a widespread lack of awareness among many globally about the dual nature of nuclear energy as both a clean energy source and a producer of formidable pollution.

In our group’s initial discussion, we recognized the diverse strengths of each member and drafted a preliminary plan for the theme. To facilitate scene setting, prop creation, and atmosphere design, we unanimously agreed on creating an underwater scenario, allowing for greater creative expression in the visual direction. This decision naturally led to the incorporation of themes related to nuclear waste water discharge. As a topic, it doesn’t have to dominate the drama but can be integrated as one of its elements. For instance, it could depict marine life struggling to return home, hindered in the last leg of their journey by nuclear wastewater, or malformed creatures born in polluted seas attempting to escape.

The strength of this theme lies in its expressiveness, particularly in sound design. The natural terror evoked by the sound of a Geiger counter can be amplified through surround sound systems, offering the audience a more profound experience. Additionally, the use of fluorescent green props in a dark space can effectively immerse the audience in the environment.

Sound of Geiger Counter

Horror Scenes

Horror scenes inherently excel in their expression through sound and lighting design. The employment of a continuous low-frequency heartbeat, sudden noises, audio spatial changes, and processed human voices can adeptly sculpt the ambiance of a performance.

Similarly, lighting control offers a multitude of options, such as sudden flickering, cool-toned lighting, fluorescent-colored scenes, and the utilization of shadows and silhouettes. These elements synergistically contribute to the creation of a profoundly immersive and unsettling atmosphere characteristic of horror settings.

The Presentation Format of the Performance

Definition of  Digital Media Performance

Traditional performance: A musical, dramatic, or other entertainment presented before an audience. Digital media performance includes virtual reality and robot performance work, telematic performances in which remote locations are linked in real time, Webcams, and online drama communities, and considers the “extratemporal” illusion created by some technological theater works.

Dixon, Steve. 2007. Digital Performance: A History of New Media in Theater, Dance, Performance Art, and Installation. Cambridge, MA: The MIT Press. https://doi.org/10.7551/mitpress/2429.001.0001.

Thoughts on Possible Format

An artistic performance can be crafted through the use of one or two backdrop curtains in combination with projected imagery. This can be complemented by handcrafted elements such as balloons, fish-shaped lanterns, or models to enhance the foreground of the stage. Simultaneously, controlled lighting or projectors from above can be synchronized with real-time audio and video animations to create an immersive atmosphere.

Possible Max Visual-Audio Solution

An example of stage arrangement consisting of a projector and foreground models

We have considered a multi-faceted approach for our stage setup, involving the use of multiple curtains, projectors, and meticulously crafted foreground models. Each scene, along with its corresponding foreground elements, is individually created and controlled by team members. The performance involves the synchronization of lighting, pre-recorded videos, sound effects, and live manipulation of foreground models, immersing the audience in various scenes and emotions. However, one drawback of this approach lies in the complexity of system setup, including sound arrangement, which can potentially lead to technical issues and synchronization challenges during live performances.

Another option I want to explore involves the use of one or two fixed screens for video projections. In this setup, some foreground models remain static while others can be interchanged during scene transitions or utilized as props by the performers. This approach offers the advantage of greater system control, reduced risk of technical glitches, and simplified synchronization of audio and visuals. It ensures a smoother and more reliable live performance experience, as sound and visuals do not interfere with each other.

These two approaches can be integrated with the previously mentioned audience-experience elements of “suspended balloons” and the pairing of light projectors, to enhance the overall effect of the live performance. Additionally, the music can be pre-arranged to include sections for expressive live instrumental performances, or to allocate certain rhythms for audience interaction, making the experience more engaging and dynamic.