Week 10

Week 10 was an intense week of change, reflection, and reprioritization, but one with a strong ending.

As we started to realize around Week 8, because of our subject matter, our project was becoming—by necessity—a research & development project, as there was a lot of testing and iteration required to really understand why many voice interactions are dissatisfying and how we could build something that avoided those pitfalls with limited technology. It was a tall order!

However, we had stumbled upon something really great back in Week 5 with only part of the knowledge we have now. Why, then, was our Week 5 demo so much stronger than out Week 9 demo? Technology was certainly a key reason: our little girl only recognized certain keywords, and in order to give our character a natural language understanding (NLU) that would give the illusion of intelligence, we had to build a new framework from the ground up. That framework is very complex and, as of Week 9, we learned we needed to restart from the ground up a third time.

Perhaps more importantly, we misunderstood what was successful in our demo. We thought the joy was in teaching, but really, it seems the delight was in discovering how to teach and then using that common knowledge.

We thought the joy was in teaching, but really, it seems the delight was in discovering how to teach.

Further Feedback & Iteration

On Monday, Anthony Daniels came to visit, which is always exciting. He took a look at both of our digital demos, and we acted out part of our new soup demo, which was not yet implemented. He enjoyed the empathy he experienced with the little girl but was frustrated because she seemed pre-programmed. As for the soup demo, Anthony didn’t accept the fact that a robot who could speak perfect English didn’t know what a tomato was—the story felt too forced. Carl, our faculty advisor, agreed and added that a) empathy might be tangential to our experience goals and b) that the story was too complex & confusing.

With this feedback and the halves feedback in mind, we set out to return to the basics of what we knew would work. The team had a fruitful debate on Tuesday as to whether or not we could get cooking to be interesting, as teaching ingredients left little room for orthogonal descriptors—for example, it would be very strange to describe a tomato as a red sphere… it’s a tomato. We also debated the merits and downsides of a fetching-centric game: it would certainly be simpler and easier, but what would our new domain be? Kitchens have a nice way of restricting the domain of what a guest would expect to be present that other environments might not.

A New Combined Cooking/Fetching Game

And, alas, a solution emerged somewhere in the middle! We decided upon a two-way fetching/serving game set in a kitchen that would provide a framework for cooking if we have the time in the future but didn’t necessitate it. The experience breaks into three beats:

  1. Greeting/introductions: We are in an alien restaurant for its first day of operation, and the player is working with an illiterate robot to serve up orders. She’s at the order window, and the player is at the pick-up window.
  2. Getting the order: The robot hands the player an order form written in an alien language, and using a translation chart on the wall, players will decipher what the code means and then teach the robot the names of alien fruits as well as the color of the required fruit. Once the robot understands, it fetches the ingredient.
  3. Serving: The robot, since she got the order, knows who it goes to. She tells the player to give it to the alien as identified by race (Gloffglorp, etc). The player needs to learn from the robot how to identify the correct alien and serve it appropriately.

We immediately playtested to see if a) the overall concept worked and b) to see if players would make up words or stick to the English domain, as our system is incapable of recognizing made-up words. You can take a look at the videos below:

Turns out that the experience was pretty fun and that, at least in these tests, there were no made-up words from the player end. With those encouraging results in mind, we decided to build out a playable prototype for our client/faculty meeting the next Tuesday.

A Technology Update

As if those changes weren’t enough, we’ve got a fairly major technology change to share. As you’ll remember, we officially put the Amazon Echo aside due to the crippling technology constraints, and we had high hopes that Amazon Lex, a new chatbot service in developer preview, might help us get around those constraints. While the system seems great, it is indeed in its very early stages, and we learned from the Lex team that the SDK required to integrate with Unity would not be released until late in Week 10 or early in Week 11. Sadly, we did not have the time to wait, so our new plan involves a bit of a patchwork:

  • Speech-to-text: Windows Dictation, as we had used previously
  • NLU: API.ai (now owned by Google)
  • Text-to-speech: TBD, but we’re looking at IBM’s Watson because of the high-quality voice and high level of emotional control

We’re off to the races with our new plan, and we’re excited since it seems to incorporate what we’ve learned and has the technology to bring it to life. Look for more updates next week!

Comments are closed.