4: Playtesting & ENGAGE Summit
On Feb 7th and Feb 8th, part of the team attended the ENGAGE PI Summit, hosted by UCLA in Los Angeles. This summit was designed to bring all the partners involved in the ENGAGE project together for presenting progress thus far, coming up with ideas for collaboration across development teams, and next steps after the summit.
The first day was a progress check with presentations from Intific/CRESST, and the ETC/HCII. Intific discussed their process and progress towards making their game for the target demographic. Their game focuses on topics under the umbrella of forces and motion, provided to them by CRESST. In addition to the science content, CRESST is also working on development in the social-emotional learning space, particularly how to react to bullying. For the ETC, we presented RumbleBlocks, Illuminate’s game from last semester. The game was met with questions regarding the pedagogical validity, which is something that we are working closely with HCII on this semester. Overall, the game was received well as a fun experience, it just needs more focus and effort into giving it more learning value. HCII’s presentation discussed how they assess children’s learning and explained their process of cognitive task analysis. They shared their findings from the cognitive task analysis and are using it to help us refine RumbleBlocks into a stronger learning tool.
The second day was focused on the next steps of the project and how to collaborate more efficiently with all the partners involved. We broke into small teams to discuss various topics/challenges of the project, such as assessment, research, and design/programming. The design/programming discussion brought many topics to the table, including user interface for this age and in-game prediction tools. The assessment team discussed different ways of getting more data from their products in order to notice patterns and trends to improve the product’s learning while the research team discussed how to bridge the gap between on-ground research and online classroom research. They also are measuring transfer from in-game activity to real-world activity. At the end of the workshop, the TA-1 teams got together with their TA-2 teams to discuss next steps. For us at the ETC, we are working towards improving the measurement of RumbleBlocks, while tackling scientific reasoning goals provided by HCII through new content. Some of these goals include constructing explanations/designing solutions and engaging from argument based on evidence.
Playtesting with RumbleBlocks & Ramps
On Thursday Feb.9th, we had a small playtest at the Children’s School on main campus. This playtest served two functions: one of which was to test the rapid prototype, Ramps, to see how well kids understood the mechanics and how (if at all) well the users could choose the right ramp. The other function was to test a new mechanic of RumbleBlocks, in which the user has to remove blocks that are not contributing to the structure’s stability. This was a suggestion brought to us last semester by HCII as another possible way of seeing whether or not the kids are learning the principles that RumbleBlocks is designed to teach.
The RAMPS prototype was well-received visually and in terms of being fun. Seven children played the game and enjoyed the look of the game. They found it fun to play, even when they were failing. Some users felt it was a bit too stale, they wanted more moments of excitement and motion to happen. There were a number of things about the prototype that didn’t work however, such as the clarity of the goal. The arrows did not work as expected; most of the users didn’t know what to do with them and some didn’t even notice they were there. They also did not seem to get the relation between the height of the ramp affecting the win/lose state.
The RumbleBlocks testing was smaller, but gave us important information nonetheless. One of the things we wanted to test was the mini-game/assessment levels. HCII suggested that we try using separate blocks for the towers instead large monoliths. We tested this in the playtest and it seemed to have worked very well. First, the kids definitely prefer the multiple blocks as opposed to the large grey ones. This is probably due to the consistency this creates from the main game. Another task we implemented was the process of removing blocks from a pre-built structure. This would give the kids an opportunity to test if they have been learning the principles from RumbleBlocks, while also providing HCII a clear “right/wrong” selection for better assessment. In the test, the users seemed to like exploring the tower after it fell: they wanted to see what would happen if they continued to remove pieces.