Week 11 was Team STEAMineer’s first playtest since Spring Break and our first Remote Playtest. This means it was also our first playtest where the team was not present for the playtest. That being said, lots of the specific information shared across the team needed to be recorded and consolidated for the benefit of our playtesters. Furthermore, it brought together elements like the game build, the instruction guide, the physical Arduino core, and the survey that were all being developed simultaneously. Individual elements were developed with an awareness of the other elements, but during Week 11 it became apparent that this “awareness” lacked familiarity. This resulted in lots of disparities. The beginning of the week saw Team STEAMineer trying to alleviate as many disparities as possible for our first remote playtest.
It is well-known that first year students at the ETC participate in the Building Virtual Worlds class. However, a lesser known part of that class is the Marshmallow challenge. Long story short, it involves constructing a tower out of unconventional materials that has to hold a regular sized marshmallow at the top. One of the things you learn in the process of construction is that marshmallows are deceiving. What seems like a light, airy confection can be rather heavy, and if you don’t accommodate the weight while you’re building, when you try to add the marshmallow to the top, the structural integrity of the tower is compromised. I bring up the Marshmallow challenge, because combining the elements of our activity into a cohesive experience was a heftier task than anticipated.
Max and Mimi worked to get the gameplay build to a point where it was stable and effective. They added in all the art elements and we even added in some of the Carnival of the Animals for background music. Mimi had started adding in extra elements to support player experimentation with the controller inputs. Different inputs would result in different on-screen effects. Mimi also worked to integrate a testing space in the game where players could see how the Arduino inputs would work for the different functions. However, it was not able to be implemented in our playtest on Friday.
We also ran into an issue with the newly-purchased hardware. The Arduino inputs were not available with pins for wires to readily connect between the inputs and the microcontroller. Corresponding cables for the inputs were purchased, but the microcontroller boards we ordered didn’t support the cables. Max was able to pinch the end of the wires in the holes of the sensor for the purposes of programming, however, the connections were still unstable. Furthermore, when the new inputs become disconnected, the whole build has to be reset. The input cannot just be plugged in again. For a more long term solution, the programming team started looking into different hardware options and the option of soldering.
Mia, Isabel, and Marissa worked on consolidating the text-based curriculum document we were preparing for Assemble into a more user-friendly instruction guide. We set it up in a powerpoint format for ease of access for guests referencing the material on their screen. We also tried to add lots of specific material to support novice playtesters. We included screenshots of the download process for implementing the build, information about how the inputs function, and supplemental ideas and questions to support facilitators. In order to make the document a reliable resource for playtesters, it required lots of information and ended up being long. The instruction book we sent to our first playtesters was also still a first draft. It still needed additional information and more consistent layout when it was incorporated into the playtest. It was hard sharing what felt like a shadow of what the instruction guide could be with our playtesters.
Mia also worked diligently on preparing the Arduino core for our playtest. Early in the week she had constructed a wooden casing for the Arduino and the six sensors. The input sensors were positioned on different sides of the core. The joystick and motor were in fixed positions on the top and front, but the button could be positioned anywhere on one side. The temperature and the light sensor were on a sliding rail on the opposite side of the controller. The sound sensor was also in a fixed position. Mia also crafted a template for a cardboard “outer shell” designed to wrap around the Arduino core. Not only would it serve as a space to mount the button, it also served as a surface for playtesters to customize their controllers. It was designed to fit around the other sensors. This way, our playtesters could decorate, theme, and personalize their controllers without damaging the Arduino core. This was important because it was the only core we had to test with since the snafu with the new hardware compatibility. We would need the core to be returned after the playtest.
Yifan and Isabel continued work on our playtest surveys. Yifan was very mindful of what kind of information we were gathering about our playtesters and what information would be meaningful to gather. With help from Dave’s input last week, they focused more on trying to understand our playtesters– like what games they play and how regularly. This was in addition to feedback about the gameplay, activity, and instruction book. Our final results included two playtest surveys: one short survey based on playtesters who would not get the full activity experience. These playtesters would see the introductory information and play the game with their keyboard, but they would not build their own controller and play the game. This survey would come in handy for the ETC’s remote Playtest Day the following week. The second survey was developed for playtesters who would participate in the full experience. Beyond collecting data about the guest and the gameplay, it asked about their experience with the instructions and building a controller.
With the Arduino core and outer shell complete, the team reached out to Dave to schedule pick-up for the hardware components for the playtest. Dave and Mimi arranged a time and location for the Arduino drop on Wednesday. We momentarily became a team of spies trying to handoff critical electronic components for world domination.
Unfortunately, there were emails to send and we all returned to our reality as graduate students trying to organize for a playtest. Information we hope to send in a prompt, single email, was sent in 3 separate emails to Dave for the playtest. (More of the information we were learning we would need to share to make playtesting more accessible for playtesters when we could not be present.) We introduced the activity, forewarned them of any supply needs and the deteriorating function of the temperature sensor, and promised a link to a Google Drive Folder that would have all the digital elements for the experience. Late Thursday night, we sent Dave a folder with the build, the first draft of the instruction guide, and the playtest survey for the Culyba Family Playtest Friday Morning. We planned to have Max on call via Slack for troubleshooting.
The Culyba family spent roughly ~50 minutes participating in our activity and filing out the survey. They were able to download and play the build with the keyboard, printed off the instruction guide, and built a controller that they were able to use to play the game. In the first playthrough with the keyboard, Hazel got 6 crystals. With the controller, they played through the game twice– on the first round Hazel only got one crystal playing by herself. On the second round, Hazel and both of her parents used the controller to collect a total of 19 crystals. All that being said, things did not run as smoothly as we would have hoped. Furthermore, the Culybas are not novices when it comes to playtesting games.
As explained earlier, the instruction guide was designed more for the sake of digital reference than printing. So I regret to inform you that they ended up printing out 30+ full-color pages. Also, referencing the material digitally is not the most practical approach as it either requires multiple digital devices to simultaneously play the game and reference the instructions, or players have to switch between the gameplay and the instruction booklet on a single device. The document was also hard to follow– especially when it was printed out into separate pages, because we included information about different parts on multiple pages instead of a single page. The book failed to warn the playtesters of cutting a hole in one side for the power cord– something they realized right before they were ready to play. The horizontal landscape did not help printing, and when asked about confidence in the instruction book as a resource, it was given the lowest rating possible at 1 out of 5.
A larger issue was the disparity between the game and the instruction book. The instruction book was making promises the game couldn’t keep. The game had problems to solve that the instruction book and controller assembly were not solving. In order to improve upon our project and make it more meaningful, we would need to more smoothly integrate these two elements. Specifically, the instruction guide suggested that players could select inputs for any function, but the build only supported selecting functions for collecting and attacking. Meanwhile, Hazel had planned out using the light sensor for the power function, in place of the motor, to remedy the issue of the motor constantly needing to be turned to keep the submarine powered. When asked if she would consider the game controller successful, Hazel said no. The Culybas shared that needing three people to use the controller felt like they were “breaking the game”. Ultimately though, they needed at least one extra person in order for it to work.
The Culyba family feedback included that turning the motor was the most frustrating part of the experience and that the mouse wheel was a better solution for the power function than the motor. The attachment to the motor to make it easier to turn also broke off in the process. This means that the controller really isn’t solving the intended issue of making controlling the submarine easier. (We wanted the controller to operate better than the keyboard inputs.) While this was frustrating for our players, it did give them a good opportunity to identify issues in the activity and where things could be improved upon. While Hazel was not able to apply her engineering skills as heavily in the controller design, she was able to apply them to the activity as a whole. It also brought up the interesting question of what it means to be making a controller. Hazel remarked that she had not touched any of the wires in the process. When asked if she would want to have that included in the experience, she declined the offer.
Dave shared that the biggest change he would want to make to the activity would be the addition of a testing space, so that players could directly see the results of how the inputs would change their activity. Without a testing space, they had to play the game blind, not knowing how the sensors might work or if the sensors would even work as intended. Especially if the problem we intended for players to be solving was to improve upon the gameplay by using a controller with inputs, having a way to test the inputs would be a good way to support players achieving this goal.
Ultimately, between watching a video of their playthrough, interviewing Hazel and Dave, and evaluating their post-playtest survey the team gained a lot of insightful feedback. We learned that the game was useable, and made sense, but the experience overall needed iterating. We also learned that the Arduino works best when its plugged into the computer directly and not through a USB hub. Positives of the experience included the attack function, which Hazel shared was her favorite part. Hazel was also able to make a Unicorn- themed controller complete with sea shells and a horn. Hazel would like to see different kinds of sea creatures, specifically dolphins, as well as a larger submarine. She would prefer the crystal collection to work by driving over the crystals instead of having to hit a key or other input. The instruction guide needed iteration, and needed to be more accessible in a non-digital format. We also received feedback on the post-playtest survey. Some questions needed clarification, and we learned some questions were not asking what we intended to be asking.
Following the post-playtest interview, and the team watching the playthrough video, we met through Zoom to discuss next steps. We were preparing for another playtest at the end of Week 12 and would need to implement changes in anticipation of the playtest. Ultimately we decided on reformatting and polishing the instruction guide, adding in a working test space to the game, and re-configuring the controller layout to make operating the inputs easier. We also decided to make the entire experience two-player. We also wrapped up the week getting materials in place for the ETC’s Remote Playtest Day event– held in place of Playtest Day which was originally scheduled for April 4th.
Learn about our activity iterations, our Remote Playtest Day feedback, and our next playtest in Week 12!