During the creative design phase of this project, we searched extensively online for reference projects. In the end, we selected some of them and developed our own design work based on them.
Image References about Performance
In these reference images, the performer and the video presentation seem to be two independent entities. However, visually appealing effects may help the performer to better convey the musical presentation to the audience, for example, by using video to help viewers understand the expression of the music. Therefore, in our performance, the presentation of the video content may be abstract, but the interaction and musical presentation can be coordinated with the image to complete the story narration.
In reference image 2, the performer’s hand and the video image or hand shadow in the image correspond to each other, bringing some stunning visual experiences to the audience. We drew inspiration from this short experimental film and realized that when many objects are repeatedly stacked in the image, it can give the audience a sense of oppression. This is similar to what we want to present, where the mother gives pressure to her daughter. At the same time, through the shadows in this work, we came up with the idea of a simple interaction between the actor and the video material. The person’s shadow in the video and the actual performer exist in two different spaces but are connected. Based on these ideas, we completed the ideation process for the third part.
Video References about Interaction
In addition to image references, we also looked for related interactive installation art videos as our references.
1. Tomás Saraceno
Firstly, we looked at the works of Tomás Saraceno. In this work, Tomas used lines to form spider-web shapes, and when the audience touches the lines, they create sound.
We believe that lines can also be represented as strings of a piano. In our performance, the mother and daughter are connected by an invisible bond, which is visualized by a red thread. If a piano wants to make a sound, it needs the vibration of the strings, which is also the vibration of the lines. Therefore, in our design, in the second part of the work, we used black and white lines to represent the piano strings. However, unlike Tomas, it is difficult for us to make the strings produce sound. Therefore, we chose to tie solid bells to the lines in this part. When the audience interacts with the lines, the sound of the bells will appear. At the same time, the positional relationship of the actors also generates random sounds through Max as a supplement to the sound of this part.
2. Particle References
We have found some reference videos on motion capture and particle effects. The interaction between dance and particle effects can bring great visual impact. However, if the visual of particles dominates the visual presentation, it may make the performer lose its functionality in our performance. Also, the interactive effect may not be better displayed, only playing a video made by particle system for the audience. In our design, to match the theme of the piano, the main visual color is black and white. In order to simplify the related visual effects, we design the color of the character’s particles as white, reducing the degree of dispersion. We also add sound control to the particles, but only make them dissipate when the sound reaches a fixed range.
3. Touch Designer
We found that most of the Touch Designer cases we found online imported pre-produced music files and analyzed them to generate video changes. For example, in our reference case, it used chop to controller and noise controller to simulate the feeling of blood spreading from paper by analyzing the emotion and style of the music. However, according to our idea, we hope that the sound can generate real-time video effects changes. Therefore, we use the Device in module of the microphone instead of audio to achieve real-time sound analysis and video changes.
