Designing the Bands
Having a guest play along with animated musicians required several components to work together in sync. First, the music. A band has individual virtual musicians, many if not all playing at the same time. The programmers on the team built the band system to handle the synchronization of various players’ sound files. NOTE: When working in development environments with sound clips which require […]
Read More
Improvisation
Teaching piano improvisation techniques and providing a space in AR for piano students to practice creating their own phrases, melodies and rhythms became the primary instructional focus of the Music Everywhere project. We were fortunate to have a variety of resources to help us develop the teaching method. We consulted with CMU faculty, professional jazz, blues, rock and classical […]
Read More
UX Exploration – Animated Hand Revisited
During playtesting, we attempted to identify the types of keyboard overlay indicators that most accurately led students to play the correct notes. More on that later… Playing back the videos from the playtests, we additionally noticed students often would play through the demo using only one or two fingers. Highlighting notes might show which notes to play, but how to play […]
Read More
UX Exploration – Avatars Revisited
As visual elements have the tendency to do, Yen’s recent depiction of a blues band using character avatars provided some much needed inspiration for the team. Moving away from animated animal characters and towards the human form, we created the beginning of a virtual band. Each member of the band plays a WAV file for the duration of […]
Read More
Remote Piano Learning
There is a version of Skype built for specifically for HoloLens. We came across it often in the early days of this project while setting up the infrastructure. This got us to thinking about how to expand the music instruction concept. Could we open up our model to give piano instructors the agency and interface to […]
Read More
Preparing for Quarters
After the semester’s fourth week, project teams at the ETC go through a process called, quarter walk-arounds and sit-downs. The faculty will stop by our project on Monday to get a brief overview of the work done thus far, and on Friday professors will spend a bit more time giving us feedback. For quarters, we are preparing the […]
Read More
UX Exploration – Avatars
An approach that came up during our brainstorming is the use of avatars. This is based on two functions. 1. Engagement and interaction In the context of piano lessons and exercises, the presence of an animated “instructor” or “tutor” is potentially useful to speak directly to the guest and explain what it is the application needs the guest to […]
Read MoreUI Development – Keyboard Overlay Iteration
We continue to experiment with variations of AR indicators in the HoloLens to demonstrate particular notes for the guest to play. As previously mentioned, our primary design challenge is working within the limited FOV (field of view). Placing a piano keyboard overlay for the entire length of the piano key often produces cut-offs from the guest’s perspective. Our first […]
Read More
UX Exploration – Animated Hand
An area our team would like to explore is the use of 3D-modeled hands to demonstrate piano concepts for the guest via AR that could potentially include aspects of proper hand positioning, posture, correct note selection, fingering, intervals, articulation, etc. The technical aspect of this involves first how to accurately generate the hand for optimal learning […]
Read More
Lesson Builder Development
To facilitate the implementation of teachable concepts into the Music Everywhere application, Phan has been working on a lesson builder environment for the Unity game engine. The tool uses a variety of nodes placed within a timeline to order sequential event-driven guest interactions. For example, the narration node will be used to give context and instructions to the guest, the environment node will provide […]
Read More