To echo the theme, the poster design primarily combines particles and text to further depict the process of memory fading away. In terms of color scheme, it predominantly features tranquil shades of blue and purple.
Category: Diary
Part 1-Dialogue fly-in animation
Brochure Design
I used Adobe Illustrator to create manuals for the five parts of the entire project. Finally print it out. It is convenient for experiencers to better understand the entire process.
Part 2-AI-generated sounds for poetry
I used AI software to create simulated human voices. The background sound of the poem we wrote in the second part was produced using AI.
This is process in AI Voice Generate
This is AI voice
Final video visual design (Part1+Part2)
As collator of the final video assets，in addition to completing my own tasks, I actively assisted teammates in resolving technical issues and coordinated visual assets to achieve better visual effects.
Adjustment of the detailing process:
- PART 1
- Step1: Emphasis on keywords
- Step2: Trailing effect
- PART 2
- Step1: Model Import attempt
- Step2: Adding transitions
Final integrated video assets (part 1 + part 2):
Part4 – Vanish Interaction Design Process
Goal：In the final stage, users have the opportunity to experience the dilemma of not being able to see their true selves in first-person perspective. The blurry human-like figures composed of particles, along with the constantly changing facial videos in the background, symbolize the fading memories of “who am I,” implying that individual lives will eventually vanish like tiny particles disappearing into the darkness.
Idea：In this section, I primarily utilized an external camera and TouchDesigner to work in tandem, capturing and recognizing faces to generate particles. Additionally, I re-edited the facial videos from the second part, gradually transitioning them from a normal blending effect to something quirky and color-changing, overlaid at the bottom. Consequently, users can vaguely discern their own faces blending into AI-generated faces during interaction, unable to clearly see themselves.
Technical details：
Overlay video：
The final interaction effect：
Part2 – Blur visualisation about the portrait
Part2 – Blur visualisation about the portrait
Previous attempts at this part（detailed in the mid-term assignment）
Final Process presentation
Idea：The long farewell in this section signifies the blurring of faces. As the memories of faces continue to blur and distort, it becomes the ultimate question of who they are and who I am.
Design Steps：This section is divided into four main production steps:
Design process and technical details：
Step1-AI Face Images
Step2-Face Fusion Video
Step3-Clear to fuzzy visual changes
Step4-Detail Adjustment
I added some jump words and transitions using mostly AE. Then, based on tutor’s advice in post, I used Capcut to overlay more abstract visual effects, such as digital streams and abstract graphics
Final video (without sound)：
Reference and project documentation link：
Presentation Mechanism Design
Our project is divided into four parts, illustrating the progressive intensification of a patient’s memory disorder symptoms, and therefore must be played in sequence. To ensure each section plays automatically during the formal presentation and minimize the need for manual control, I designed a Max Patcher to handle the playback of each part, as illustrated below.
The presentation of our project is split into four parts to show the progressive deterioration of a patient’s memory disorder symptoms:
1. Video Playback (Parts 1 & 2): Utilizes Max’s jit.playlist which outputs a “done” signal after a video finishes, automatically triggering the next video. This ensures a seamless, sequential playback without manual intervention.
2. Audio-Visual Interaction (Part 3): Employs Vizzie for audio-visual interaction, triggered by a toggle that activates both microphone and camera capture upon receiving the “done” signal from the previous video.
3. Interactive Segment (Part 4): Transitioned via a manual click due to the variable duration of Part 3. Developed in TouchDesigner, it employs the NDI protocol to send video signals to Max on the same computer, allowing for control over the display’s start and end within Max.
4. Loop Initiation: A Bang button is designed to close Part 4 and trigger Part 1’s video, starting the cycle anew. This setup ensures that all parts are played in order and that transitions between different technologies and formats are smooth.
Hardware Control
To help the audience quickly understand how to engage with the exhibit, I mapped all operations to hardware controls and created a guidance display for the computer screen. I used the Korg nanoKontrol and selected two buttons for this purpose. This setup aims to simplify interaction and enhance user experience during the exhibition.
Hardware Setup
Screen Recording
A Max patcher screen recording for a complete presentation loop.
Final Present System
Week10 Research on AI, Voice and Memory
Voice is an important part of memory, especially including the emotions of the speaker. It helps people recall memories of their loved ones. When using TTS models, we find that at the current level of development, AI-generated audio can already capture the emotions contained in the voice very well. This is also why we decided to use AI-generated voice. Below are some research topics about AI, voice, and memory.
The relationship between human voice and memory is profound, serving as a crucial element in both preserving personal histories and evoking emotional responses. Studies and projects, such as those explored by the Oral History Society and in digital archives like Voice Gems, illustrate the emotional and historical significance of recorded voices. They underline how voices not only act as a personal echo from the past but also as an emotional trigger that can bring memories vividly back to life.
Scientific research supports this emotional connection. Studies on recognition memory for voices highlight that the human voice can act as a powerful memory cue, influencing how memories are encoded and recalled. These findings are crucial for understanding how auditory elements of memories affect our recall and emotional responses. This interplay between voice and memory is not only significant for personal reminiscence but also plays a vital role in creative expressions, where voice recordings are used to create impactful art and preserve cultural heritage.
For further insights:
- Voice and Memory Projects
- Voice Gems Article
- Recording Voices
- Recognition Memory for Voices
- Memory Cues Study
