Educational Goals

Who are we?

ENGAGE is a collaboration between education specialists and game designers at Carnegie Mellon University's Human-Computer Interface Institute (HCII) and Entertainment Technology Center (ETC). The goal is to integrate instructional design and game design to produce highly innovative educational games for science learning plus scientific evidence of their effectiveness.

These games will be the basis for reproducible innovations in the instrumentation of educational games, including how to design games to be diagnosticof student learning and socio-emotional development, and how to make educational games adaptive to learner's developing knowledge state and collaborative skill.

This new knowledge will be based on a strong set of experimental studies of educational games, comparing it against a number of contrasting control conditions. Experimental evaluations of educational games are still too few and far between. Given the enormous interest in educational games, scientific evidence related to their effectiveness is urgently needed.

Virtual balance beam used for cognitive task analysis with children

Education Specialists

The team of education specialists brings a wealth of expertise in human-computer interaction, psychology, linguistics, and advanced technologies and includes experts in game design, scientifically-based instructional design, and educational research with children in pre-Kindergarten (four years old) through grade 3 (eight years old). They have had great success in iterative development and field-based evaluations and dissemination of advanced learning technologies including mobile and web-based games, intelligent tutoring systems, and computer supported collaborative learning environments. One team member has already developed an award-winning educational game.

Team members also have a great history of successful collaborations in creating a technology-based infrastructure for educational technology R&D; NSF funding of the Pittsburgh Science of Learning Center (PSLC) has reached $35M. The education specialists at Carnegie Mellon's Human-Computer Interaction Institute also have a history of successful collaboration with three other key partners on this project, Carnegie Mellon's Entertainment Technology Center, Carnegie Mellon's Children's School, and Sesame Workshop.

Instructional Design and Game Design

In addressing challenges specific to educational games, the education specialists bring cognitive task analysis and instructional design into the game development process. Previously, methodologies for creating games did not provide the specific guidance desired to integrate learning objectives and game play objectives in mutually supportive ways. The team created a flexible framework for the design and analysis of educational games that helps with this critical aspect of educational game design. The education specialists work with the game developers to apply key elements of the framework to make sure that the game's mechanics, dynamics, and aesthetics work to support the game's educational and socio-emotional objectives according to established instructional design principles. As part of this effort, the education specialists work with the game designers to make each game maximally diagnostic. Retrofitting educational assessment into games not specifically designed for this purpose is difficult; designing games with this goal in mind right from the start is much more likely to be effective.

DataShop

One key part in the education specialist's evaluation of educational games is DataShop. Part of PSLC's infrastructure, Datashop is an on-line facility that supports automated, fine-grained data collection and analysis of learner interactions with tutoring software and educational games. DataShop has captured more than 60 million data points from more than 45 thousand learners across the U.S. DataShop allows the education specialists an unprecedented ability to evaluate game designs by capturing detailed, moment-by-moment data about how well the game supports student learning, individual engagement, and social engagement. Each game creates highly detailed logs of player interaction. Using DataShop, the education specialists can semi-automatically analyze the game's log data to uncover the nature of the skills (e.g., their level of generality and potential for transfer) that students acquire in the game by fitting different skill models to the log data.

Socio-emotional Learning

By applying data mining technology to natural language, gestural, and other acoustic data recorded while children play the game, the education specialist team monitors social and emotional engagement with the game and between game players. These data provide timely and detailed insight to the game developers as to how well the game supports learning and socio-emotional engagement, so that they can make timely adaptations.

Adaptable Games

The education specialists help make the games adapt to differences in student learning by applying their proven knowledge-tracing and cognitive mastery technology. These technologies have been repeatedly demonstrated as effective, are in widespread use, and are continuously being improved through the collaboration of cognitive scientists and machine learning researchers in the PSLC.

Iterative Evaluation and Use-Driven Design

Timeline for formative evaluation of RumbleBlocks game

Timeline for formative evaluation of RumbleBlocks game

In addition to single child or small-group play testing, the team of education specialists conducts frequent formative evaluations with groups of 50 or more students. These studies provide frequent feedback on student learning, engagement, and other outcomes (e.g., socio-emotional learning) to the game developers. Other studies provide feedback on how well the game strikes a balance between fun and learning in comparison to games that emphasize either fun or learning.

Results of research and updates on development are available.