Project Pupil goal is to create prototypes to imagine Augmented Reality (AR) interaction / design patterns for future classrooms. This means that at a low-level, we will explore guest’s gesture language inside the future classroom, and at a high-level, we will find an AR interaction that enriches classroom learning. Before using boards or computers as education tools, students learned a subject by observing, feeling, and discussing real objects. These days, the main interaction has shifted from the to a 2D mode through paper or electronic devices. We seek to explore the return of education to the physical space, and discover design patterns and affordances to create pedagogically sound prototypes.
We are exploring three types of classroom interactions using AR technology.
The first is learner-content interaction – how does a student’s interaction with subject matter take place in AR?
The second is learner-learner interaction – how can AR improve collaboration among students?
The third is learner-teacher interaction – how can AR elevate a teacher’s capabilities in the classroom?
Our goal is to identify patterns in AR design across these different interactions in the classroom for our client and other researchers in the space to expand on.
Initially, we considered using the Hololens, Meta 2, and the Magic Leap. as our head mounted AR platform. However, these platforms have limitations in their field-of-view and hand tracking quality, and thus were not a good fit with our vision of a future AR headset.
Therefore, we decided to use pass-through VR as a means for AR, with a Leap Motion to get access to high-quality hand tracking and interactions.
We chose to start with learner content interaction, as it lends itself to a single-player experience. Prototyping in this domain involves exploring a variety of student interactions with virtual content – so we had to answer the major question of what platform would be able to support us in that goal.
For our first round of prototyping, we focused on a different strength of AR, but were kept simple to quickly test the viability of our passthrough VR headset. The first prototype focuses on 3D content visualization by having users observe vertex coordinates and modifying a line in 3D space through hand gestures.
For our second round of prototyping, we used layer model. The reasoning behind using layer model is when we pitched our idea of a layered model for teaching biomes to a group of teachers, they pointed out that it would be very useful to have a similar model to teach archaeology. We realized that layers are a repeating pattern across more than one subjects, so we decided to implement an interactive layer model to explore those interactions.
For our third round of prototyping, we will focus more on classroom interactions between students and between student and teacher. We decided to proceed with a prototype in the space of chemistry for the content. We are looking at creating a task-based learning method for the VSEPR theory (of molecular structure). The interactions for the prototype are: teacher demonstrate the theory, students share their understanding, and finally submit their understanding to the teacher.
Check out our development blog
Check out our design document
If you have any questions about our project, please contact