As researchers trying to understand games, it’s invigorating – and humbling – to see the breakneck pace with which game development occurs. Every few years, revolutions in hardware, design innovations, and the changing place of video games in culture transform the marketplace in fundamental ways. A few years ago the revolution was games’ capacity for meaningful narrative experiences. Then we saw the enormous growth of MMOs. Today games are reaching new audiences through social games on emerging platforms. Games are a moving target, and understanding them means incorporating many points of view on a changing basis.
We are designers and academics who cross boundaries, and we value how engaging with each other enables us to reflect on our practices, encounter new ways to think about games, and see how other fields tackle similar problems. RTR grew out of this impulse for interdisciplinary dialog. Organic conversations at conferences such as the Game Developer’s Conference, or Games + Learning + Society, occur most often during spontaneous dialog over dinner or martinis. Over time, these discussions led to further informal and formal collaborations. These include academics studying game developers’ design practices, game designers conducting guest lectures and teaching game design courses, and both groups consulting on one another’s work. On occasion, full-blown collaborative projects sought to push the envelope of academics and game design, as with Gamestar Mechanic, a game originally developed by the University of Wisconsin-Madison and Gamelab, led by James Paul Gee and Eric Zimmerman.
As useful and productive as these efforts have been, we wanted to create a space to promote such interactions, without necessarily requiring one to close down the bar or have a large grant. Building on ideas from games design, could we pull from the tradition of prototyping, and create a quick and easy cycles of learning, and could we pull from learning theory and create contexts to learn through problem solving?
“We always talk about game designers and academics collaborating in designing games,” Eric noted, “but why not have them collaborate in conducting research?” This struck Constance and Kurt as a little weird. Why would game designers, who create these compelling experiences and luscious worlds give two shakes about research?
But the more we thought about it, the more collaborating on conducting research made sense. For starters, it might enable discussions of “what is a good research question?” Academics might be intrigued by what kinds of research questions game designers have, and it might be helpful for game designers to go through the process of creating and refining research questions so that they’re clear, building on theory, and answerable. Thinking through the problem might enable academics and designers to share what each knows about games. For example, participants might share information on gamer demographics and play patterns, formal or informal theories of player motivation, or theories of design.
The opportunity to pursue these ideas presented itself through the Games + Learning + Society Conference. Each year, we promote unique formats that honor the principle of “learning through interaction,” such as “chat-n-frags”, fireside chats, and design workshops. The idea behind all of these formats is to move away from content-delivery as the model for conferences, and toward structured interactions that are likely to produce learning for participants.
Through these discussions, Real-Time Research (RTR) was born. What started as a structure to facilitate learning through interaction has evolved into an intriguing research format in its own right. In RTR, people (ideally they are interdisciplinary and from different fields or industries altogether) gather to conceptualize, conduct, complete, and report out on a research study within a very brief (usually 2-3 day) time period. This might sound insane (it did to us at first), but it works, in no small part due to the structures and supports that RTR facilitators provide (see next section). You might think of RTR as the rapid prototyping of research, although with rapid prototyping there is usually an implicit goal to build a larger product later on. In contrast, the process itself, as a learning experience, is the primary goal of RTR.
Over time, RTR has evolved to take advantage of the unique opportunities that this form of research allows. At a conference such as GDC, 10,000 game designers -- and players -- of many different sorts gather, forming a unique population to be studied via any variety of means (observations, interviews, surveys, structured experiments). Likewise, conferences, which occupy physical and virtual space in particular kinds of ways create new opportunities for social interactions. RTR experiments provided some of the inspiration for games such as Backchannel, a conference-based game played over Twitter, that was later expanded by Zimmerman, Colleen Macklin (an RTR veteran) and colleagues.
These are just some of the opportunities that RTR provides. With this volume, we are turning RTR over to you, the reader, player, researcher, and designer. Our hope is that RTR will morph and evolve as people adapt it to new contexts and domains. The body of research within, although usually containing low evidence for generalizability is nonetheless useful, either for gaining insight into gaming as a social practice (see studies of World of Warcraft players’ inventories), prototyping new methodologies (see post-it note studies), or capturing state-of-the-field at particular times (see Wordle studies of the Game Developer’s Conference). We imagine that as this corpus of RTR grows and evolves, one may be able to query it to gain insight into “what the field was thinking” through time.
Before your RTR Session
As you prepare for your session, a few important considerations. First, the overall schedule. We have run RTR with the following general structure:
An initial session to explain the process, divide the researchers into groups, and design the research experiments. This session requires a room with breakout tables for group design discussions and lasts from 90 minutes to two hours.
After this session, researchers are left on their own to meet and organize their research projects as they see fit during the larger event.
Lastly, the researchers meet again at the end to finalize and present their research, as well as discuss the overall process. This last session could be as short as an hour if they are just presenting, but we recommend 90 minutes or more so that groups can spend some time preparing their final presentations.
If you are running RTR at a conference that lasts a couple of days to a week, run your first session as early as possible in the event, and run your second session as close to the end as you can. That gives your researchers as much time as possible to meet and work on their projects during the event.
If you are running RTR in a context that is less time-condensed, such as within a class that meets regularly, you can simply hold the beginning and ending sessions during class hours. Because we have only held RTR sessions at conferences, we’ll be aiming our tips and suggestions for that kind of context.
Who is running your session? We’ve had good luck with groups of 20-30 researchers being led by three or four facilitators that represent different disciplinary backgrounds. Ideally, you have the people that are:
from an academic background and familiar with a wide array of research methods.
comfortable with data analysis and multiple theoretical approaches to game studies.
with a design or instructional background who is used to working to solve design problems through rapid prototyping.
It’s hard to find all of this in just one or two people, so we highly recommend a team approach to facilitation. Facilitators should not be participants – they need to run the starting and ending sessions, help groups with the initial design process, and provide assistance throughout the process.
There are a number of materials you will need to run your session. If you want to run your RTR event as we have done, here’s what is required (everything is explained in more detail in later sections below):
• RTR Cards (Tool 1, page 170)
3 decks of cards for THEORY, METHOD, and TOPIC.
• Supplies & Session Prep (Tool 2, page 175)
To assist in planning and implementing projects, including large pads of paper, markers, post-it notes, etc.
• Goodies (Tool 3, page 179)
This is not mandatory, but to assist researchers in recruiting subjects, we have provided candy treats and special “I subjected” stickers for conference attendee badges.
• Template Slides for Group Presen- tations (Tool 4, page 181)
• Research Documents & Handouts
(Tools 5–8, page 182)
We’ve provided sample questionnaires and data forms, human-subject interview guidelines, and tips for research. All of which are available for you to modify at the end of this book in the section titled “THE TOOLS”.
• Headquarters & Resources.
While participants are actually working on their projects during the conference, it is nice to have a high-traffic location where they can set up shop to conduct experiments or recruit research subjects. Our best results came from conferences where we had a large “Real-Time Research” sign near a table where researchers could leave surveys and goodies, set up posters for interactive research feedback, etc.
In addition to the office and art materials mentioned above, some great resources for researchers during the conference would be a place where they can easily print or photocopy documents like a survey form, access to still cameras and video cameras, clipboards, and – of course – more art and office supplies! In our experience, if you provide them, the researchers will use them.
RTR is not “just another session” at a conference, but is more like an event that is woven into the entire conference from beginning to end. For this reason, we highly recommend that you work with the event organizers to try and secure the time slots, locations, and spaces you need in order to make your RTR event a success. Being able to have a prominent RTR headquarters, for example, that includes a printer and other resources can really help out your researchers.
Promotion is also important for getting the word out to possible participants at the conference. You don’t want people to hear about how cool the first RTR session was after it happens – you want them attending! Since your opening session will be taking place at the very start of the conference, you need to make sure that people know about it. If possible, try and sell Real-Time Research as a “special event” that should be promoted as such at the conference. Perhaps the final session of research presentations can be given a prominent spot in the conference program.
Opening Session: Structure
The breakdown of the opening session is as follows, assuming you have 90 minutes total. Be strict with your time! You have a lot to squeeze in.
Introduction by the facilitators: 10 minutes
Dividing into groups: 5 minutes
Handing out cards: 5 minutes
Swapping cards and finalizing card selections: 10 minutes
Brainstorming research questions and experiments: 20 minutes
Rapid-fire pitches and critiques: 15 minutes
Final implementation planning: 25 minutes
Each of these are explained in more detail below.
Opening Session: Intro
OK. Your preparation is completed, and you’re ready to run your first session. To begin, introduce the idea of Real-Time Research to the room. In addition to summarizing the process for everyone, we recommend that you hit the following important points:
THIS IS NOT TRADITIONAL RESEARCH
Set their expectations properly – Real-Time Research is almost certainly not going to produce top-notch research results. But that’s not the point. The purpose of RTR is to collaborate across disciplines in a playful way, as they explore new research methods and approaches. And who knows – they might end up with some real insights. But no one should go into an RTR session expecting the rigor of traditional academic research.
RTR IS A COMMITMENT
Taking part in Real-Time Research is not a back-seat experience. It means rolling up your sleeves and doing something – not just for this session, but for the rest of the conference. Everyone at a conference is already probably quite busy, and doing a Real-Time Research experiment means that you have yet another set of tasks to squeeze in. So give them a chance to switch to another session if they like – they will need to be able to set aside the time to do their research.
IT WILL BE FUN
Even though RTR research will be work, it will be play as well. Participants have fun, get stimulated, and often end up with new projects or even ideas for publishable papers. Furthermore, RTR is great networking. Not only will participants collaborate with people in their research group, but they’ll have an excuse to approach anyone at the conference and ask them to take part in their study.
In your introductory remarks, find a balance between scaring them away and encouraging them to stay. You don’t want half-hearted participants: if people flake out, it is tough on the rest of the group. On the other hand, it may take some convincing to get your session attendees to see what is so great about staying in the room and committing to the experience.
Getting everyone into groups is the next item on your agenda. Groups should include five or six participants. In our experience, fewer than that number and a group might not end up with enough person-power to complete a research project (especially if one or two drop out). With more than six in a group, it’s easier to take a back seat and not end up really engaged with the group discussions and decisions.
Just to be sure you know who you’re dealing with, you might want to ask people to raise hands based on their home discipline (design, humanities, social science, technology, education, etc). Your hope is that each group has a good mix. Because people from the same background who know each other tend to sit together, we have found that “counting off” works best to shuffle the room. Figure out how many groups you will have (such as four), and then go around the room, counting 1-2-3-4, 1-2-3-4, etc. until everyone is in a group. Each group forms around a different table or area of the room.
We have put together a worksheet, evolved through several iterations, that will help each group plan and implement their project. Practically, this worksheet asks participants to ‘commit’ by adding their e-mail for group communication. In addition it is useful to have each group’s leader fill out the worksheet (Tool 5, page _) and then hand it in to you at the end of the first session, as it not only helps them see their ideas evolve, but also serves as a record of what they have accomplished at the end.
Dealing & Swapping Cards
Next comes the fun part: deciding what each group will research. In a wildly interdisciplinary group, giving participants a blank canvas would be a disaster. To help them coalesce quickly around a single idea, we have used a set of cards (Tool 1, page _) to help them shape their ideas. Constraints help foster creativity, and the cards we present here are the result of trying out and tweaking of structures that will shape innovative group thinking.
Each group is dealt two Theory of Learning Cards (Jean Piaget, Behaviorism, etc.), two Topic Cards (Play Styles, Second Life, etc.); and one Methods Card (Observational Studies, Interviews, etc.). The goal is for each group to come up with a viable research idea that takes one of each kind of card into account. To give a sense of how these constraints get turned into projects, we present the original cards given each group on the first page of each group’s chapter.
As groups are looking over their cards and beginning to discuss them, lay out all of the undealt cards face-up on a table in the center of the room. Let groups know that they can send a single representative to swap cards they were dealt with cards on the table. That is, as long as they follow The Swap Rule: you must lay one card down on the table before you pick one up. This guideline is important to ensure that someone doesn’t swoop in and swipe all of the cards.
Don’t give groups very much time to finalize their cards – ten minutes at the most. Expect heated discussion as groups debate the cards they want to keep, while representatives scurry to and from the central table. In our experience, some groups will take good advantage of the swap table, but other groups always end up using the original cards they were given. By the end of the card swapping time, each group needs to have finalized the three cards they want to use as the basis of their research experiment.
As groups begin to generate ideas, remind them that time is of the essence. The most important thing for them to keep in mind is that they must quickly move from bouncing loose ideas around to picking a single concept for their research experiment. They will have time to refine their idea during the rest of the session, but it’s important for them to be decisive rather than deliberating endlessly.
Facilitators should feel free to wander by groups, listen in, and give suggestions. Give them room to breathe, but push them if they need it. As a way of structuring their thinking, try having each group formalize their research question – as well as the real-world experiment that will attempt to answer that question.
Remember to encourage them to be playful in their methods – this is their chance to go for unconventional research techniques. Do they invent a game that is played by everyone during tomorrow’s lunch? Take a survey by setting up posters with instructions for self-reporting? Can the results of their research end up being a wall-length mural? A collaboratively written story? A video puppet show? Use examples of RTR projects from this book to help them see how open the possibilities really are.
Note that the groups do not have to be orthodox about fully using all three cards – perhaps one of their cards is more of a tangential inspiration than a hard constraint. The most important goal for them at this stage is to rapidly find consensus around a single research idea.
Discussion and Critique
After about 20 minutes have passed, it’s time for a quick discussion and critique. Even with their concepts at such an early stage, groups must present their ideas to each other for feedback. Have them present their research question, and then outline the experiment that they want to perform.
Having discussion and critique so early in the process serves a number of purposes. The fact of having to present helps put pressure on groups to be decisive and settle on an idea. A healthy sense of competition among the groups can also be a motivating factor. The notion that their concept is getting critical feedback keeps everyone thinking fast and loose, and open to change and improvisation. Lastly, of course, groups will always have useful feedback for each other too.
We have structured the critique in a few different ways. Sometimes we’ve paired groups up with each other, so that each group hears and critiques one other group. We’ve also had each group pitch their concept to everyone, getting feedback from the entire room. Both work well - time and space constraints will help determine how you want to structure this part of your session.
As groups present, facilitators should ask questions and give comments. Make sure that each group is asking an original, interesting research question that takes good advantage of their interdisciplinary mix – it shouldn’t sound too much like research from any one narrow field. Feasibility of implementation is also a crucial issue – keep an eye out for groups that are proposing projects requiring too much time and attention from them or from their research subjects. Can they really get it done in the time allotted?
Once the presentations have been completed, it’s time for the final stretch – each group needs to plan concretely how they are going to implement their experiment. Here’s where you really need to help them strategize about how they are going to accomplish their research. Typically, RTR participants underestimate just how busy and distracted everyone is at the conference. For example, if they are going to put up an interactive poster to gather research, make sure the instructions are dead clear, and if possible set up shifts of researchers to stand next to it. If the group wants to observe people doing a particular activity – like playing games – find out where at the conference people will be playing them and ensure that the researchers get there at the right time.
Make sure that everyone knows how things are going to unfold after the session ends. That means communicating the time and place of the final session at the end of the conference, as well as everything that needs to happen in-between. Designate a group leader to collect email addresses and mobile numbers from everyone. Make sure that each group knows where and when they are meeting to begin their actual research process.
Communicate to the groups all of the resources they have available to them. This includes physical art and office supplies, facilitators who can be reached to assist groups in getting other materials, the location of an RTR headquarters where groups can print and photocopy, goodies for test subjects, consent forms, and anything else that you have put together in preparation for the session. Let groups make unusual requests and see what you can do. Make use of the hive mind: if RTR hasn’t reserved a video camera, someone in the room just might have one.
As they exit the room at the end of the session, make sure there is a leader for each group with a contact list, as well as a hard plan for how they are somehow going to manage to conduct a research experiment in the time that unfolds before the final session.
During the Conference
Once the session ends and your researchers scatter to the four corners of the conference, do what you can to support them in their efforts. Below are some of the strategies we have taken in past RTRs, some of which work better than others in particular contexts.
1. CREATE A SUPPORT PERSON.
Whether this is one of the facilitators or a conference staff, have someone that every researcher can call with questions and requests. Make sure this person really has time to answer phone calls and emails, as well as actually meet with and help out groups that need assistance.
2. HAVE A GENERAL RTR MEETING TIME.
In the past, it has helped to have a suggested daily check-in time and place for groups to gather and touch base. This is especially useful in large and busy conferences, where improvised meeting times may be difficult or impossible for groups to make. Ideally, your meeting times take place daily during conference down times. And make sure your support person is there to help out.
3. CREATE A REAL-TIME RESEARCH HQ.
If you can manage it, having a central table where researchers can gather can be very helpful in many ways. An RTR HQ can be a meeting place, the location of RTR resources, and the spot where the support person can be found during most of the conference. An HQ also serves as a rallying point for actual research – it can attract attention and therefore possible research subjects. (You can let interested any test subjects know about the time and location of the final session, where they get to see the results of the research they’re facilitating.) In addition, this can serve to advertise for your next RTR event when people stop to ask questions.
4. GIVE GUIDANCE AND SUPPORT.
When you see RTR researchers in action, stop to ask them how things are going and tell them how much you’re looking forward to their final report. Feel free to offer any feedback or discuss their preliminary findings or methodologies.
Closing Session: Wrap-up
The final session should be simple and focused. The main purpose is to let all of the researchers tell their war stories and – hopefully – share some interesting research results. Depending on the length of time and format of your event, you may want to give time for researchers to finalize their presentations – say, the first half hour of a 90-minute session. On the other hand, if you are pressed for time, and are expecting lots of non-researchers to attend the session (who may not want to wait half an hour for the presentations to start), then tell your groups to show up with their presentations ready to go.
Most likely, each group will only have a short time for presentation and discussion. Encourage them to keep their slideshows and talks short, and let the details of their experience come out in the Q&A. A PowerPoint Template (Tool 4, _) is included in this book, which you can copy onto laptops at the opening session in order to facilitate and structure researcher presentations.
Ideally, each group presents the RTR cards they decided to use as inspiration, their research question and experiment, the process they lived through trying to complete their experiment, and any results and tentative conclusions. If you plan on doing RTR again, asking participants how the experience could have been better for them is a good idea.
Post-RTR: Contact us!
RTR is a passion for us and we hope that it will be for you too. Our work with it is only the beginning of the fun. We believe that your efforts to use it will produce the same sort of experiences. Try it out and enjoy it. If you do, let us know!
We are more than willing to work with you. At the least we want to hear about your experience informally. There are two ways to share. Contact Seann for either planning and personalizing your lesson design or just to trade war stories. Or you can have your groups write up their research for review - using the format (Tool 6, page _) you see in this book - and send it to us. We’d be excited to see your modifications, your group’s final work, possibly invite you to add a chapter to this book, and to welcome you to the RTR community.