Resources

Press

Calendar

Orientation

Directions

Similar Programs

Map of the ETC

Multimedia At Work: Entertainment Technology Center
Donald Marinelli, Randy Pausch, and Janeen LaForce, Carnegie Mellon University

There's no doubt that the world of digital entertainment is here to stay. In addition, ongoing advances in computer technology will continue to expand not only the graphics pallette of the medium but also the realm of imaginative, virtual experiences being offered to consumers. At Carnegie Mellon University's Entertainment Technology Center, we focus on creating new entertainment technologies, venues, and experiences that take advantage of developments in technical fields like computer science, while seeking ways to include the traditional dynamics of fine arts such as drama, storytelling, film, and theater.

As educators of entertainment technology, we're continually asked to define our discipline. Our mission statement is to provide "leadership in education and research that combines technology and fine arts to create new processes, tools, and vision for storytelling and entertainment."

Mixing arts and sciences

In 1998 we set the wheels in motion to create the Entertainment Technology Center (ETC) at CMU, which has top-ranked technology and fine arts programs on the same campus. You're probably aware of the reputation of CMU's School of Computer Science but you may not be aware that CMU was also the first university to grant an undergraduate degree in drama (1914).

We created the ETC as a direct initiative from the university president. A two-year program of study, the ETC culminates in the Master of Entertainment Technology (MET) degree, which is jointly conferred by the College of Fine Arts and the School of Computer Science. One of the directors (Randy Pausch) is a professor of computer science, human-computer interaction, and design. The other (Don Marinelli) is a professor of drama and arts management.

The curriculum centers on interdisciplinary group project work but also draws on courses from the breadth of the university's schedule (see "Building Virtual Worlds"). The ETC doesn't intend to turn artists into engineers, or vice versa. Rather, the program seeks to create new professionals with an understanding of how to use technology to entertain and inspire. While some students will achieve mastery in both areas, the program intends for typical students to enter with knowledge or experience in one specific area and spend their two years at CMU learning the vocabulary, values, and working patterns of the other discipline. We're based on the principle of having technologists and fine artists work together on projects that produce artifacts intended to entertain, inform, inspire, or otherwise affect an audience/guest/player/participant. In short, the program will produce graduates with unique, cross-disciplinary skills in technology and fine arts. These students can, and have, been paired with industry partners on specific sponsored projects. 

The ETC works with a variety of industry partners to transform how people use and interact with technology. In addition to pushing the envelope of what's traditionally considered entertainment, we also apply the principles of entertainment, via new technological capabilities, into broad areas-from developing customer loyalty and branding in the digital era to training geographically dispersed workers. 

Students in the ETC take courses ranging from computer programming to designing virtual worlds to improvisational acting, but the emphasis is on project courses. Each project course places students in interdisciplinary teams that must produce working artifacts; in the tradition of CMU, this emphasis is on making real things that work. The project courses progress by slowly increasing the duration of the projects, which start at two weeks and ultimately last for an entire semester. A key aspect of the program is to ensure that the students have an opportunity to work with a large, diverse set of collaborators with different skill sets. To this end, the project groups are constantly reshuffled during the course of the students' four semesters with the ETC.
 
Student projects

Although the program has only just finished its pilot year, our students have already produced the following projects.

Content production

During the first semester of the program, the ETC was invited to work with a San Francisco Bay Area location-based entertainment company to create content for a virtual-world, intergalactic travel experience that the company had under development. Our students created a series of news shows for the fictitious Inter-Planetary News Network. From concept to design, they modeled the cybernews desk and other locations, wrote the news stories, and produced several complete newscast segments. The theory was to make the news stories and other content accessible to users through a series of video touch screens throughout the facility.

Training "game" for network administrators

Being prepared to deal with network security is an increasing problem. Many administrators are ill equipped to handle network attacks, and training can be dry and boring. Since many of these administrators are avid game players, the CyberSecurity Center, a division of the Carnegie Mellon Research Institute, recruited the ETC to design a compelling, game-like training tool. In the first phase of this project, the ETC assisted with concept and content development and eventually began proof-of-concept prototyping. We hope that this ongoing project will result in an interactive game that simulates hackers breaking into networks.

Virtual reality rat maze

To what extent can a "God's eye view" and a set of controls let external viewers alter the choices players make in an immersive environment? ETC students set out to discover the answer. External "directors" sitting at desktop machines that display an overhead view of a maze attempt to lure the VR participant/explorer by cuing real-time audio and visual changes to the environment. They also perform distortions such as warping the environment (Figure 1) to make certain rooms appear closer, larger, more brightly lit, and so on. Our students found a director could change the story on-the-fly in an immersive simulation and dynamically draw the participant's focus.

Figure 1. An example of how an external director can change the
VR maze environment in an attempt to control the players' actions.
Would you be lured in the direction of the bulging door?

Audience control of dinosaur time machines

One of the earliest partnerships the ETC developed was with the Carnegie Museum of Natural History, located adjacent to campus. The museum contains a partial-dome SkySkan theater, which uses five digital projectors on a 210-degrees wide by 30-degrees tall wraparound screen that envelops an audience of about 60 people with full-motion imagery and surround sound.

While the theater was not built with real-time interactivity in mind, ETC students immediately set out to create just that. What they came up with-in only six weeks-was an interactive experience geared for school-age children. The premise was simple but effective: the audience would be transported to a prehistoric landscape in a flying time machine and, by controlling the direction of the machine, they could complete their mission of taking snapshots of various dinosaurs they encountered. 

Because our students were working with the museum, their challenge was to create an experience that was not only entertaining, but also educational. In the spirit of scientific accuracy, they formed the flying time machine in the image of a large Pterodactyl (technically, a Quetzalcoatlus). As the narrator of the piece explained to the audience, the dinosaurs they were about to encounter would not be frightened by a time machine that looked like one of their own.

The experience was fully interactive; the audience controlled the direction of the machine through its flight as a cohesive unit. Rather than using indirect, artificial controls (such as buttons in the armrest), this experience used direct, full-body motion. ETC students created a real-time, computer-vision system that tracked the audience-by leaning to the left or right in their seats, the audience collectively controlled the flight path of the virtual time machine through a prehistoric canyon. (Imagine in Star Wars an X-wing fight flying through the Death Star trench, except that 65 people are piloting it.) While flying through the landscape, the guests were surrounded by full-motion imagery. They took pictures of the various dinosaurs they encountered, and discovered interesting
dinosaur facts from a real-time, synthetically generated narrator.

ETC students built the entire system using consumer-level commodity hardware and software. For the vision machine, they used a SAG electronics 700-MHz Pentium III and an Imagenation PXC200 video capture card. To build the rendering machine, they started with a Dell 866-MHz Pentium III with one Diamond Multimedia Viper 770 accelerated graphics port (AGP) card (for the control monitor) and added five 3dfx Voodoo3 peripheral component interconnect (PCI) graphics cards to run the five screens. Students connected the two machines by Ethernet. Additionally, they used an in-house 3D rapid-prototyping tool, called Alice, to create the prehistoric environment.

When beginning this project, our students set out to discover whether the audience would behave as a collective. They discovered that their audience (several troops of Junior Girl Scouts) displayed excellent cohesion in decision making-the creative mix of storytelling and technology carefully woven by a small group of graduate students encouraged an audience of strangers to interact with each other almost immediately. This experience was a success. However, the jury is still out on how well this technology will work long term. This semester, our students continue to develop and research audience-controlled interactive experiences for an immersive theater in a museum environment. 


The CUBE: An inside-out CAVE

A surrounding, immersive environment can be provided in two primary ways: a head-mounted display (HMD) or the Cave Automatic Virtual Environment (CAVE). The CAVE is a cubic room typically about ten feet on a side where rear-projectors display images on three, four, five, or six of the surfaces. The ETC Computer-driven Upper Body Environment (CUBE) takes a new approach. Imagine the top half of a phone booth hanging in space (Figure 2). Viewers duck under and into this device, which provides imagery on all sides. Like a CAVE, the user need not wear or be touched by anything, but like an HMD, it requires very little floor space. Unlike either, the CUBE gives a set of surrounding viewers a unique show of the participant's activity. After constructing the CUBE, the team presented Swamp Romp, a virtual reality world created by one of the team's members in a previous semester's Building Virtual Reality course. The world lets guests play the part of a frog, jumping from lily pad to lily pad in search of flies. The team chose Swamp Romp as the demonstration world because the circular lily pad path encourages guests to turn 360 degrees and view all four screens. Specific human-centered observations from the project include:

Figure 2. A model of the CUBE, developed by ETC students in an
attempt to create an immersive environment similar to a CAVE in
that no equipment must be worn, but like an HMD, takes up very
little floor space.

Potential uses for the CUBE include telepresence, vehicle simulation, immersive display authoring environments, and general exploration of 3D virtual reality worlds. To learn how to build a CUBE, see "CUBE Equipment: Make Your Own at Home."

The students in our pilot class have just completed their first year of research, study, and professional development-resulting in the artifacts, concepts, and projects just described. As we admit a new class and await the graduation of our pilot class in the spring of 2001, we find the ETC not only riding the wave of a new media revolution, but also creating waves of its own. Join us often by logging on to our Web site. 

Readers may contact the authors at the Entertainment Technology Center, Carnegie Mellon University, Doherty Hall 4301, 5000 Forbes Ave., Pittsburgh, PA 15213, e-mail Marinelli at thedon@cmu.edu, Pausch at pausch@cs.cmu.edu, and LaForce at jlaforce@cs.cmu.edu.

 

Building Virtual Worlds


The ETC offers a number of unique courses, including Building Virtual Worlds. This course puts students into four-person teams, where each team consists of students from four different majors and/or skill sets. The teams use a combination of commercial and locally developed software (available at http://www.alice.org) to produce a fully functional, HMD-based virtual reality world in less than two weeks (Figure A). At the end of each two-week period, the teams are shuffled, to maximize working with as many different people as possible, and the process repeats. By the end of the semester, each student will be part of five teams, making five completely different VR worlds. At the end of the semester, the class holds a campus-wide exhibition to show their work, attended by a standing-room only crowd of more than 500 people. Many of the worlds are available on the ETC Web site.



Figure A. One of the worlds created in the Building Virtual Worlds
class was a simulation of a Disney ride. The students created a
world in which the ride stopped running unless the participants
sang "It's a Small World."

CUBE Equipment: Make Your Own at Home

Our students set out to create the CUBE using creativity and forethought. In fact, no advanced building skills were required-just the ability to use a drill!

Here's the hardware you need to get started:


To make the screens and frame, you'll need: