Postmortems

Introduction and overview of the project

Mobius Master is an engaging and effective experience to help refresh technicians about how to use bio-pharma equipment made by Merck KGaA (EMD). These technicians already took the onsite training session at the beginning, so they have a general idea about how to use the equipment. However, after several months without practicing, they are not familiar with the operation process anymore. Our goal is to create an easily accessible refresh training method for technicians all over the world, overcoming some constraints in traditional refresh training sessions.

We designed three different game to cover the content of refresher training.

Find it is an virtual reality game using Google Cardboard to help operators get familiar with the structure of the Mobius Bioreactor. In the VR worlds, players will be given tasks to find target components, and get corresponding score based on the correctness and efficiency of their performance.

Sort it is an augmented reality game to help operators get familiar with the sequence of steps about how to operate the mobius bioreactor. Players will get a deck of cards. Each cards indicates one step of the operating process. Player will use Sort It game to help sort out the right order of the nine steps, and get corresponding score based on their performance.

Choose it is a traditional quiz app to help operators get familiar with the detail knowledge about how to operate the mobius bioreactor. In this app, players will need to answer a lot of questions in different format, such as multiple choices question, true or false question and fill the blank questions.

What went well

1. Customer requirements analysis

We did a lot of background research in this project in order to know what our client really wants and needs.

We searched for many different training methods that are available now using different technologies, and came up with a brief idea for each method so that client can have a choice. The ideas include training using HoloLens, HTC Vive, Amazon Echo and etc.

On the need side, we interviewed the engineer who designed the machine and is in charge of the training session of it over Skype. Besides Yiheng and Carsten went to Boston to try the bioreactor by themselves and talk with technicians who know how to operate the equipment face to face.

2. Technology

Taking into consideration that our client wants something that is easy to setup and accessible. We decided to take a combination of augmented reality and virtual reality using mobile device. We use printed cards and smart phones to do augmented reality and Google cardboard to do virtual reality. All the materials and devices are really easy to get and cheap. With detailed user instruction, even people with any tech background can set up everything easily and use the app by themselves.

3. Play testing design and analysis

We hope to measure if our experience would improve or deteriorate the learning effectiveness of our users in different aspects, and how much would it be. According to the correctness of each aspect, the priority of the learning objectives could be analyzed as reference for future development. Besides data collection, contextual inquiry would also be an important part. Observation during the demographic are testing the experience would help to ensure the usability of our VR or AR experience.

We talked with Mike and Jessica Hammer asking a lot of play test design advice. Also since Yiheng was studying physics, he used to do a lot of experiments. He contributed a lot of professional testing methods such as A-B test and double blind test. So I believe our play testing process is very standardized.

4. Iterations based on play testing

Through continual play testing, we found a lot of places that we can improve, so we made a lot of UI/UX changes based on the testing result. For example we had several camera shape portals for players to change point of view. However, after play testing, we found that most people do not know what does it mean. The problem was fixed by changing the portal model from camera shape to arrow shape so that the function of the portal is more clear. We did a lot of improvements like this to make our game more user friendly.

What could have been better

1. Communication with the client

Before the quarter, we had difficulties communicating with our client. He told me he was OK with us doing anything, but the reality is whenever we bring up an idea to him, he was not satisfied with it. This problem bothered us long time, our process was delayed by this. Later our solution is to list out a lot of possible solutions for him to choose and also talk with expertise who designed the equipment to figure out the real needs from the demographic perspective.

2. Communication with faculties

During soft openings, a lot of faculties did not get the point of our project. They did not understand why certain decisions were made and why we did not choose other technologies. We made every decision after careful consideration and we have adequate reasons. However we did not communicate well with faculties during the soft, which make them feel confused.

3. Assign the roles early

There are five people in our team — four programmers and one artist. At the early stage, in our team, specific role assignments were messy. No one had producing or design experience. We tried to figure out these tasks together. However, we realized that it does not work. The efficiency in this way is extremely low. Then we started to redefine everyone’s role and it works, but it would be better if we had this thing earlier.

4. Play testing with demographics

As the project was very specific to its demographics : technicians who used bioreactor before, it was very hard for the team to arrange play tests with specific demographics.

Our solution was to test with common play testers to test the UI and UX. Then we switched the bioreactor content with content about how to use coffee machine. We can sort of get the result we need by this way, yet it’s still better if we could play test with real demographics.

What we learned?

1. It is hard to make a clear instruction for a complex project, but it is very important.
2. Due to the different background the team and clients may share, we should pay more attention that there could be some different understanding about some concepts between us and the client. For example, our client always address that he wants our app to be gamified. Actually gamification in his mind is totally different from gamification in our mind. He does not mean that he wants us to make a game for him with nice game design, however he just want this experience to be more sticky and engaging.
3. Sometimes clients don’t know what they need too. Clients usually want everything, which is not possible. So it is important for teams to work with clients together to figure out what they really need and prioritize those elements.
4. Define specific roles in a team as early as possible. It can help team members know their responsibility and improve efficiency.