Preparing for Quarters
After the semester’s fourth week, project teams at the ETC go through a process called, quarter walk-arounds and sit-downs. The faculty will stop by our project on Monday to get a brief overview of the work done thus far, and on Friday professors will spend a bit more time giving us feedback. For quarters, we are preparing the […]
Read More
UX Exploration – Avatars
An approach that came up during our brainstorming is the use of avatars. This is based on two functions. 1. Engagement and interaction In the context of piano lessons and exercises, the presence of an animated “instructor” or “tutor” is potentially useful to speak directly to the guest and explain what it is the application needs the guest to […]
Read MoreUI Development – Keyboard Overlay Iteration
We continue to experiment with variations of AR indicators in the HoloLens to demonstrate particular notes for the guest to play. As previously mentioned, our primary design challenge is working within the limited FOV (field of view). Placing a piano keyboard overlay for the entire length of the piano key often produces cut-offs from the guest’s perspective. Our first […]
Read More
UX Exploration – Animated Hand
An area our team would like to explore is the use of 3D-modeled hands to demonstrate piano concepts for the guest via AR that could potentially include aspects of proper hand positioning, posture, correct note selection, fingering, intervals, articulation, etc. The technical aspect of this involves first how to accurately generate the hand for optimal learning […]
Read More
Lesson Builder Development
To facilitate the implementation of teachable concepts into the Music Everywhere application, Phan has been working on a lesson builder environment for the Unity game engine. The tool uses a variety of nodes placed within a timeline to order sequential event-driven guest interactions. For example, the narration node will be used to give context and instructions to the guest, the environment node will provide […]
Read More
Paper Prototype 3
Phan’s lesson builder is in the testing phase, and we’ll likely transfer these skill-building overlay prototypes into a holographic environment. However, these have been useful for a variety of reasons: • Guests can learn in abstraction, without thinking that they’re learning transferable piano skills. Especially those that are at times considered tedious—like scales and arpeggios—this separation has led to […]
Read More
Score Module Development
As we prepare for the addition of a score object in the AR space, we’re working on two means of creating and working with musical notation. a graphical editor for simple note entry directly within Unity a parser for the musicXML format The commonly used music notation software, Sibelius, can export a piano score in a variety of formats. The […]
Read More
UI Development – Keyboard Overlay
Effective UI design will take experimentation and testing. Direct and pervasive key overlays pose problems of occlusion or guests not being able to see their hands. The limited field of view that HoloLens can use is a design challenge. As well, HoloLens’ FOV is intended for the guest to look straight ahead, not necessarily down at a piano […]
Read More
MIDI Over Bluetooth – Implementation 2
The Roland FP-30 digital piano, like the Korg microKey Air, can send MIDI data over bluetooth. Similar to the 25-note microKey, bluetooth pairing with the Windows workstations was unsuccessful. Pairing with HoloLens, on the other hand, was practically automatic. Having an instrument with full-sized piano keys allows us to develop, observe and test on the intended device, an electric piano. The […]
Read More
Paper Prototype 2
Left-Hand Finger Exercise: All-white key piano model created with construction paper and post-its. • A continuation of version one, the guest is now presented with a different set of markers on which to play with their thumb, using their left hand. Starting this time with their fifth finger and working towards their thumb, this hand and […]
Read More