Michelle B. Xu
City Scent: Real-Time Interactive Synesthetic Performance
Motion Capture ｜ AI｜Multisensory XR Theater
City Scent explores the journey of a scent molecule as it transforms through human perception and imagination. This interactive performance combines audience participation, AI, and neuro-sensory data to create a multisensory experience.
Upon arrival, each audience member receives a scented paper infused with a unique perfume. Inspired by the fragrance, they are invited to describe a scene on their mobile phone. This input is then translated by AI into real-time visuals projected behind a dancer. The performance deepens as a spectator wearing an EEG sensor shares their neural response to the scent, with brainwave data dynamically influencing the visual effects. This creates a living connection between the dancer, the audience, and the scent.
City Scent is a poetic interplay of scent, movement, and technology, offering a narrative where every participant contributes to the unfolding story.
Nov 2023-
Feb 2024
-
EEG Sensor
- Unreal Engine
- Optitrack Motion Capture System
- Big Led Screen
- Storytelling/Design
- Live Motion Capture
- AI scene generation
- Signal Processing
- 3D world building
- IoT-Based Wireless Communication
- NYC Media Lab * Chanel Synesthesia Challenge, Best Journey Award, New York City, Navy Yard
Credit:
Lead Artist/Technologist: Michelle B. Xu, Wei Wu
Unreal: Yoki Ding
Dancer: Justine Kaneda
Story
How much time passes before an aromatic molecule fades away?
When released into the air, an aromatic molecule absorbs moisture voraciously, eventually dissolving and vanishing into the atmosphere, unseen yet eternal.
The performer embodies the essence of an aroma molecule, using her movements to express the shifts in scent over 10 minutes—interpreting the life cycle of these invisible particles.
Simultaneously, I will capture a volunteer’s brainwaves, mapping their fluctuations as the scent evolves. These brainwave patterns will influence the performer’s scenarios and her avatar, creating a dynamic interplay between scent, mind, and movement.
Experiemental Design
Technical Details
1.Brainwave Sensor + Unreal
2.Data Visualization + Body Movement
Performance 2024