TheatAR Project Brings Childhood Classic to Life with AR

This story originally ran on CMU’s news site

By Laura Kelly Email Media Inquiries

All it takes is faith, trust, a little pixie dust and augmented reality to bring Tinker Bell to stage like never before.

TheatAR, a semester-long student initiated project in the Entertainment Technology Center (ETC), combines stage actors and animation to create a first-of-its-kind experience in the retelling of the classic “Peter Pan.” Through a Microsoft HoloLens, an animated Tinker Bell interacts with Peter and Wendy upon their first meeting, bringing her mischief and mannerisms once reserved for movies to a theater audience.

“Peter Pan is a beloved story that pretty much everyone has a familiarity with. It allows us to showcase a very beloved character, Tinker Bell, that flies around and interacts with people,” said Dan Wolpow, a graduate student in the ETC and creative director and producer of the project. “We thought this familiar storyline would be a great gateway to show what this technology can do.

“For the first time ever in a production of Peter Pan, [the audience] will see Tinker Bell not just as a flashlight on stage or a spotlight. She’s going to have a three-dimensional presence. She has a face, she has a body, she has body language, and we’re going to be able to watch her actually interact with the actors on the stage,” Wolpow said.

To make Tinker Bell into a 3D character, Wolpow turned to his co-producer and animator, Euna Park.

“For Tinker Bell, I actually looked at a lot of reference videos of hummingbirds — the way they float and hover in an interesting and surreal way. I also combined that with the way the human body moves because obviously Tinker Bell is not a bird,” Park said.

Park also incorporated actions seen in indoor vertical skydiving to complete the aesthetic.

“There are videos of it on YouTube, and I didn’t know it existed. That’s part of what I love about animation,” Park said. “There are things I don’t know and I can discover them to use as reference.”

Audience members will see Tinker Bell either through the HoloLens AR headset, or by watching TV screens on either side of the stage, which will show what the HALO Lens is seeing. However, the actors, CMU drama students Amara Pedroso Saquel and Will Harrison, rely on hitting their marks and lighting cues provided by Raisa Chowdhury, the physical interactions designer, to ‘see’ where Tinker Bell is on set.

“A lot of this project has been lighting design for special moments when Tink interacts with the world. She has her own glow, and so if she’s hiding behind an object, she can’t just disappear. There has to be some indication that she is still present,” Chowdhury said.

Programmers Sudha Raghavan and Apoorva Ramesh then pull the physical and virtual worlds together to give the audience the total experience.

“All four HoloLenses are networked together. You launch one HoloLens first, and it shares the whole stage dimensions with the other HoloLenses,” Ramesh said. “Once I launch the application in the first HoloLens and register the whole stage and scan the whole surface, this data is shared with the other lenses so they will get all of the stage dimensions directly from the first. This way, they all are connected and are seeing the same thing, but they can see it from their own perspective.”

Working on programming projects like this is what brought Raghavan to Carnegie Mellon.

“I honestly could not find any program in the world that would help me focus on doing AR/VR as much as the Entertainment Technology Center would. That’s the reason why I wanted to do this graduate program and applied to Carnegie Mellon in the first place,” Raghavan said.

---