Week 15 – Post-Mortem

Introduction

Commit to the Bits is a discovery project exploring the intersection of improvisational theater and Twitch live streaming. By applying factors unique to Twitch audiences such as their members’ mutual awareness and their potential for dramatic agency, we designed performative frameworks that embrace the character work, active decision-making, and role-playing inherent to improv. Our deliverables are a set of novel user interaction prototypes exploring hypotheses that arise from this design space and documentation of new design principles for it. We primarily worked toward these through methodical playtesting, rapid prototyping, and qualitative and quantitative data analysis over fifteen weeks in Spring 2019.

Foundation

We began by mapping out various parts of each medium. We read roughly 50 research papers on topics relevant to improvisational theater and emerging design ideas on live streaming platforms. Using knowledge gained from this literature review, we drew up diagrams illustrating our understanding of how a live streaming and improv combination may be approached, including this!

We needed to make necessary assumptions of our users to produce prototypes. While we identified pros and cons about audiences of different skill levels, we eventually determined that our audience should be comprised of streamers highly-skilled in improv performing for a Twitch audience with average improv experience and knowledge. This choice was made because we predicted it would both have the largest potential audience with the highest potential for quality entertainment.

To narrow our scope, we focused on three primary qualities:

Interaction Types

Voting

Voting was used in Freeze Tag and to an extent in Music to my Peers. This was the simplest starting point for the project as it already exists in Twitch experiences such as Choice Chamber and Twitch Plays Pokemon. It was easy to aggregate audience inputs literally via polling.

Audience members voted on peer submissions in Freeze Tag while the scenes were running. At the end of a scene, tallied answers were displayed for a time, before the voting session repeated. This segmented voting pattern allowed for a series of rising and lowering tension, but we did not capitalize on this much.

Surprisingly, less viewers voted on content than submitted it. Of active viewers, 67% voted during these two games.  This was a good first step into the space, but we potentially spent too much time on it.

Audience Submissions

Audience Submissions are a common interaction in improv acting. Improvisers usually ask for a suggestion, based around a theme or some question, to begin their scenes with. Sometimes they develop dialogues with audience members to draw out more ideas. We wanted to both translate this system directly to understand improv better, and approach it in a way impossible to do on an average stage.

Text Submissions were our direct translation of traditional solicitation of audience suggestions, appearing in Freeze Tag and Guesstination Unknown.

Image Submissions were more novel, appearing in Whose Bag Is It Anyway? Viewers were given a Google Drive link and allowed to upload any image they liked. These were curated behind the scenes so no inappropriate content was shown on stream.  We believe this was a high barrier of entry, and was something we could’ve done better.

83% of active users submitted either text or an image throughout any applicable game, making this surprisingly effective.  We strived for a feeling of realistic potential to impact the scene, and the majority of these active users felt that. This was our largest success.

Guest Stars

Guest Stars were a maximization of audience involvement. In Discourt, audience members literally joined the scene by calling in via a VoIP program (Discord). This method bypassed Twitch’s audio/video delay by bringing it down to around 0.5 seconds and allowed guest stars the entire range of communication possible verbally.

This made viewers feel a greater responsibility to entertain other viewers. Furthermore, it required an additional account beyond just a Twitch account.  Both of these created a large barrier to entry. In our playtest, one of our viewers attempted to call in, but was stumped by the interface of Discord. We ended up having to call in ourselves to keep the show going, a bad judgement call.

Music To My Peers also included a milder element of guest starring. By revealing to the rest of the viewers who the selected DJ for the game was, viewers could communicate with them in chat and influence their decision for which song to pick.

Guessing

Guessing was an interaction made to distance our project from our other interactions as much as possible. Relative novelty was the driving force behind its appearance in Guesstination Unknown, but not much else was acquired from this interaction.

General Interaction Reflections

Much of our general feedback was that audience members wanted more control over what was going on in every game. While the majority (83%) of our audience felt they had a realistic potential to impact or control the scene, they all wanted more breadth to their inputs.  It will be challenging to balance the limitations of direct control with the inherent broadcast delay, but we are confident there are interesting and novel ways to design around delay while giving the audience at the very least the illusion of direct control and very best full control.

When brainstorming these interaction types, we often approached them from either the live streaming or the improv side of things.  This was close-minded. For the first half of the semester, we had not thought of voting as much more than the easiest way to have audience interactions.  While interviewing a professional improviser, we asked him for his most wanted feature: it was voting. Polling an audience may not be innovative on Twitch, but is unique in live theater.  From then on, we still tried to consider where any interaction type would be innovating in, but always considering what it could offer to both.

What Could Have Been Better

Securing a consistent and attentive viewership was difficult for a remote experience, especially with regards to collecting feedback. We did this by sending feedback surveys via email, but our responses were inconsistent. Future teams should leverage other resources they have at their disposal to get a constant number of playtesters (e.g. main campus), rather than rely on word of mouth.

Having a more robust knowledge of full-stack web development would have sped up development, since Twitch extensions are essentially web applications. Future teams should have programmers who are more familiar with web development processes and architecture in order for the prototype to be more extensible.  We will include a sketch of a better potential architecture in our full documentation to pass on.

Crowd feel was a large trouble point throughout the semester.  It is difficult to have contagious laughter when no one is sitting next to you, and it is hard for actors to receive feedback.  The largest problem that future teams will have is having a proper and natural dialogue between the audience and performers that creates a collaborative social atmosphere.  Our work is only the beginning with this, and needs a lot of work to finish, but we’re confident we have a strong baselines of what interactions need to be supported by a good social underpinning.

For a more in-depth view of our project, please view our full report.

Week 14 – Pre-Post-Mortem

In lieu of a proper blogpost, know that we are hard at work on summarizing our knowledge from throughout the semester in a variety of forms. We are currently preparing a master post-mortem to derive future documentation from. We expect this large chunk of everything to be file down into separate documents for recommendations to future teams, abstracts for paper submissions, and data analysis reports. Here is the introduction and segment list of our post-mortem.

Commit to the Bits is a discovery project exploring the intersection of improvisational theater and Twitch live streaming. By applying factors unique to Twitch audiences such as their members’ mutual awareness and their potential for dramatic agency, we designed performative frameworks that embrace the character work, active decision-making, and role-playing inherent to improv. Our deliverables are a set of novel user interaction prototypes exploring hypotheses that arise from this design space and documentation of new design principles for it. We primarily worked toward these through methodical playtesting, rapid prototyping, and qualitative and quantitative data analysis over fifteen weeks in Spring 2019.

  1. Introduction
  2. Prototypes
  3. Interaction Types
  4. Host
  5. Crowd Feel
  6. Barriers to Entry
  7. Technical Limitations
  8. Physical Setup
  9. Future Work

Week 13 – Finishing Touches

We pushed out our final prototype within schedule and with better features than expected, concluding our content production phase on a high note. Our final game implemented a new user interaction type: guessing. We wanted to give back to the viewers and let them do something else with a scene other than indirectly influence it.

Audience members submit a noun, verb, or location at any time throughout the game. A scene is then randomly constructed from all user submitted words by one actor. They act it out indirectly while speaking and improvising, similar to charades, while audience members guess words in chat. If a player submitted a word, they are made aware it is in play and barred from guessing it.

By limiting user submissions to certain types of words, in this case the format of “a NOUN VERBing in LOCATION” we wanted to reduce the stress of interaction. We don’t think this was incredibly successful, and we had a lot of outlier entrants. Verbs were often not in progressive tense, raw locations were hard to improvise on, and so on. While we believe this got us more submissions than a blank text box would’ve, it made it harder on the improvisers and for a repetitive game.

We also chose to have guessing happen in chat instead of on the extension anonymously. This was done to get more conversations, any at all, happening in chat. It isn’t entirely possible to consider this a success since every playtesting group is different, but we did have conversations spring up in chat based on what other audience members were guessing. This will need further analysis and recommendations once we go over our chat logs more thoroughly.

And with all of that, we’ve finished our content production for the semester. Next up is gathering up what we’ve learned to provide a firm base for future researchers, students, and streamers to continue on our work in this intersection.

Week 12 – Calm before the Stream

This week was set-up for our final crunch time before the end of our project. Luckily, it was also Carnegie Mellon’s carnival, so it was nice to get a few days off in the sun.

Trace led our brainstorming session, implementing Jessica Hammer’s methodologies

Our strongest focus was one last ideation push to see if we could complete one or two additional prototypes for the end of the year. We did a surprising job and generated many ideas for both our work and future work. As part of our final documentation, we’ll outline some of our design ideas from this for potential future work. As for what we picked, we focused in on novel interactions: a guessing game and a recurring game. We’d previously only worked in more passive forms of audience interaction, such as suggestions and voting, and wanted to better test the limits. We hope to flesh these out over the next week.

We are very aware of our impending deadlines, with production on video trailers and content summaries already begun. As mentioned, one of the most important things about our project is to inform future designers. We are laying the groundwork to best communicate our ideas and knowledge the week after next, Week 14. We hope to really dig into that process then, and maintain rigor. Until then, we’ll polish off our remaining prototypes.

Week 11 – Playtest Day

Playtest Day stream show

We were lucky enough to take part in the ETC’s Playtest Day. We gathered everyone up in a room with premade Twitch accounts and put on a brief show to fit within schedule. The audience was younger than our usual playtest group, mostly teenagers, and had a fun time. Most of our results confirmed some of our assumptions and theories, and it was nice to see people actually using Twitch chat, especially those who used it frequently outside of playtesting. They wanted more control of mixing and matching content, enjoyed hearing laughter in the room, and went on “trends” of certain content paths, like making every situation about McDonald’s. This session was short and sweet.

In terms of content produced, we also had a very smooth ideation to implementation week for a music game. In this game, a special audience member will be chosen as a “DJ” for a round of an improv scene. They will solicit (or choose on their own) three broad-genre music pieces to play over the scene to change the mood. Examples include Romance, Heavy Metal, and Sad music. We want to explore a) how audience members respond to “captains” being designated the leader for a short while and b) more indirect ways to influence scenes. Internal playtesting of this seems to have gone well, and audio balance work made sure its enjoyable on the viewer’s end as well. Time will tell when we get to playtest this.

In terms of scope, we are beginning to realize how near the end will be. We’ve been in content production mode for the last couple of weeks, and need to step back and organize our thoughts soon. We believe we’ll have one more game to be made over the course of the next week and a half before committing ourselves to soft openings, final design documentation, and sharing our work with the world.

Week 10 – Discord in the Court

As mentioned during halves, our goal for the remainder of the semester is to produce and test one prototype a week featuring a new user interaction.  Various roadblocks happened this week that gave us some insight on how this will go. Regardless, we managed to make and playtest our Courtroom game by Sunday.

The game’s main idea was to get audience members to call in over Discord and become star witnesses in a trial.  Two improvisors would act as prosecutor and defendant respectively. The former would draw and guide information from the guest witnesses, while the latter would take the accusations and weave a narrative from them.

We wanted to investigate two things with this game: a) the willingness of audience members to contribute on a greater level and b) bypassing video delay by calling in directly.  The delay bypass worked out great, and seems to be an effective form of direct communication. We further had animated characters on a green screen behind the actors to ease their communication with someone who wasn’t there.  The generic model for each guest facilitated variety. We puppeted it via preset animations and keybindings, bypassing delay further and letting us have more control of the scene’s feel.

Audience willingness and the barrier of entry was another challenge, and one we didn’t succeed at.  Only one person attempted to call in, and they abandoned the game before they were even on the line.  Our team filled in the gaps as callers, and that flow was fine in the game, albeit a bit underdeveloped due to not practicing much with our actors beforehand.  We believe this will go better in communities where there are established Discord back-channels to ease tech use or larger groups of friends to encourage involvement on this scale.

We also managed to retest two of our previous games, Whose Bag? and Freeze Tag.

Whose Bag? went near flawlessly, and was a lot of fun for everyone involved.  Our production value could be improved a bit, but this was the most satisfying game.

We moved away from any non-green screen background to ease shifting between games.  This nice, neutral brick wall “comedy club” background seemed to work fine. However, Freeze Tag had a major bug that prevented us from testing much with it.  We fixed it quickly, but felt it better to move on to our other games with the time we had.

Overall, it was nice to get back to work this week after spring break and GDC19.  We pushed out a prototype in time to test and learned a lot from it. We’ll have more reflections on it after getting all of our user feedback in order, and another prototype done by next weekend as well.

Our chaotic stream set-up


Week 9 – GDC19

The majority of the team attended the 2019 Game Developers Conference in San Francisco, and did not progress much on the project. We’ll bring you more updates next week!

Week 8 – Halves

As a team of all first years, we were nervous going into our first ever ETC presentation. But, the preparation for the showcase of our work gave us value insight into everything we’d done throughout the semester. It was good to lay out everything with the goal of understanding.

We also had to properly present both Twitch and improv to an audience that we couldn’t assume knew the depths of either. Talking about why each medium was interesting and their individual unique qualities was a good refresher on why they make sense when put together. Explaining the challenges and potential of their union reminded us of our excitement working on it. Plus, it was a good time to show off all of our graphs and images. We summed it up in the form of similarities and differences, ending on how we believe it is important to create experiences that stay true to both mediums.

We needed experiences that promoted community creation, had predictable formats, and had potential for audience impact. But, we knew we had to be wary of losing viewer understanding, intra-audience interaction, and Twitch’s inherent delay.

While we had all of these large qualities to value, we had to scope down for the remaining semester. So, we will head into the rest of spring trying to maximize feelings of audience impact potential, intra-audience interaction, and viewer understanding. To make these more concrete, we’ll be focusing down into specific game interactions and mechanics that support and explore these areas.

After reviewing our prototypes, we showed off some of our future brainstorming for the semester, and wrapped up the presentation. With our focus now firmly on prototyping, we hope to get a lot of data the rest of the project. We’re hopefully soon to be certified to undertake ethical and reviewed research as well, potentially making our findings more useful and more backed up. We’ll see how we’ll take these summarized learnings to heart as we begin making experiences more rapidly.

Week 7 – New York, New York

Our first playtest immediately after last week’s blog post was a great success.  We met and meshed well with our actors, had a decent audience size of 17, and plenty of user interactions to reference.  We playtested in two large chunks of about half an hour each, first of Freeze Tag and the second of the TSA game (rebranded as “Whose Bag Is It Anyway?”, now less copyrighted but still legally ambiguous ).  The goals of this playtest were to simply test our flow of gold spiked tech, work with our great actors, and get a base level of competency in running these shows live, and we accomplished all of that. We now also recognize the coming challenges of getting large enough audiences to make our tests worthwhile, and how best to use our future qualitative and quantitative data.

We backed away from this relatively simple playtest wanting to stretch our abilities even further.  This playtest only effectively had voting as a mechanic, and we weren’t here to limit ourselves to that.  We brainstormed ways to spread out more meaningful decision making among the performers and audience both.

The most engaging idea we came up with was a courtroom game.  It is still under construction, both literally its backdrop and its design, but the overall idea of it is to have “guest stars” pulled from the audience as witnesses on stand in a trial.  Two performers will take the roles of the defendant and prosecutor to elicit random stories from the witness’ testimonies. We are unsure of how best to guide this, and how best to technically get audience members up on “stage,” but pursuing the idea of them as more involved guest stars with performers steering the show is an appealing idea, and one we are thinking about a lot.

After a week of playtest retrospective and presentation preparation, we were able to go to New York on a research trip for the project.  Our goal was to see as much improv as possible, interview actors, and understand its conventions fully. We saw five shows in three theaters over two nights in the city, ranging from the best to the worst improvisational acting we have all ever seen.

The one that stood out the most was an improvised musical.  Not very relevant to our project’s interaction focus, but the raw skill of the improvisers showed us how engaging they can be on their own.  It seems obvious that good actors can act well, but it was a good reminder that skill and showmanship can make almost any performance. This was a part of shifting our focus slightly to include more of the crowd experience, proper production value, and beginnings, breathers, and ends in our live streams, but far from the only factor.

Speaking with the actors after the show lead to good talks.  Charlie Todd of The Curfew was our most extensive conversation, and stressed how hard it is to make virtual crowds feel like they’re in on the same “inside joke” as improv audiences are.  This struck us as a bit off. Where he was coming from was that all the improv is happening once, and never again, based on conversations, responses, and laughters from the crowd. But this sounded quite a bit like Twitch’s meme culture.  Either way, it will be important to us to facilitate the callbacks, responses, and most of all forms of language that are seen and heard in a theater but not in Twitch’s strictly text and emoticon chat.

Finally, we posted improv surveys around the three theaters we attended, with permission.  Shoutouts to the one person who filled it in. We will refine this method to get a better feel of improv audiences.