An Endless Poetry: VR Live Musical
This is a VR musical adapted from original Chinese musical An Endless Poetry.
With Optitrack, we combined the live motion capture with virtual environment and made a musical experience inside a poetry world.
The audience puts on the headset and steps into the world of two poets with actors singing around. They are inside the story, nothing is real but everything is real.
The system includes live motion capture with Optitrack, stage cue controller tool developed in Unity, which also controlled VFX to show and disappear, and Meta SDK.
🌟 Credit
Director & Dramaturg:
Motion Capture:
Character Modelling:
Environment Modelling:
Unity Developer:
Peiheng Cai
Ruyin Zhang ｜ Cuiyi Lin
Zhuoya Wang
Wenhui Xia
Peiheng Cai ｜ Cuiyi Lin
My Contribution
🎬 Directing & Dramaturg
I led the overall design part to adapt original stage musical to an immersive VR one. To fully use the advantage of virtual environment, we choose to add VFX and abstract environment to show the world of poem, which is hard to show in real theatre.
According to the VFX we designed, performers need to do certain actions to align with the VFX. I designed the blockings and actions of the two performers consistent with the virtual environment in headset and VFX.
🛠️ Stage Cue Controller Tool
The cue controller tool is very important for stage technicians in theatre production, which allows sound technicians to play backtrack and SFX at certain time in order, and allows light technicians to control the stage light change according to the script. It also allows stage manager to control the process and give cues to different departments after the show begins, so that a play can run smoothly.
For VR musical, every visual effect is inside virtual environments, which means the cue control tool should be inserted inside the engine.
I created a small tool to control all the cues including different VFXs change, light change, and playing backtrack music.
🕴️ Motion Capture
Motion capture is done under the Optitrack motion capture system. We used the Motion to capture motion data and connect to Optritrack component, making the character movement driven by the real-time capture data.
💥 Visual Effect
Fire VFX is developed with Visual Effect Graph, including the fire circle on the floor and the rising fire in a circle shape.
Nebula effect is composed of many different particle systems, using multiple textures made in Photoshop. By changing the position, scale and other attributes of particle system, texts and stars together form the special nebula effect.
