Games User Research Summit Recap

by Phil McGuinness
In one of those rare and fine occurrences where work and play intersect, I found myself at the Games User Research Summit at the Sony Computer Entertainment facility in San Mateo on March 3rd, 2015. I had heard about the conference from another usability professional, and thought it would be interesting to see how usability practices are applied by professionals in the video gaming industry. Although we live in an age of specialization, I find that it’s important to step back from time to time and look at things from a different angle. What unique challenges are faced by usability professionals in video gaming? How do they approach similar tasks, like recruiting representative candidates and designing a product on a rapid time scale? These were the questions I wanted to answer, and a few of the talks that I attended shed light on these key topics.

20150303_181419

Figure 1: Despite a typo in “McGuinness,” this pass let me mingle with the 200+ professionals at the conference.


One of the first talks of the day was held remotely by Bill Gardner from Xperienced points due to unforeseen circumstances. In an experience much like gaming online with friends across the country, Bill presented remotely via video, coordinating with the local staff to change slides when necessary.                 Bill spoke of an experience working on the game Swat 4. In the game, you control a SWAT team member from the first person, and perform missions using authentic SWAT tactics. The development team made an initial assumption about the player’s movement speed, keeping it low so as to mimic the controlled actions of a real life SWAT team. Maps were designed on a scale according to the movement speed as the game was developed.
When the team went into usability testing late in the development process, they found out that the testers wanted their players to move faster, that they felt sluggish and slow. However, at this point changing the movement speed would result in sweeping changes to the level design, and there was no time in the development process to make the adjustments. Had they tested earlier in the process, these changes would not have had such drastic implications. This is a good example of how testing early and throughout design can save resources and avoid pitfalls later on.
20150303_174524

Figure 2: User Researchers and gaming fans watch Celia Hodent speak about Usability Heuristics in the main conference room.


Another talk that piqued my interest was a panel led by Klayton Vorlick from Sony, and with Karl Steiner, Nathan Cook, and Steven Shulmister. The panel touched on recruiting for niche markets, finding the right incentive, and fielding externally. Additionally, the panel covered challenges recruiting candidates in the video game industry and shared lessons that can apply to all industries. One problem faced by recruiters everywhere, is that when money is involved, people will do whatever they can to be in your study. One novel approach to this issue was touched on by Steven from VGMarket; when Steven’s company calls potential gamers, rather than asking a list of standard questions, they use recruiters who are knowledgeable about the subject to ask pointed questions that verify knowledge. For instance, when a user claims that they have played a specific game, recruiters will ask who their favorite character or boss battle was. This provides a checking process while screening that isn’t always applied in a traditional approach.
The final talk I’ll cover here was presented by Eric Hazan, regarding Games User Research practices at Ubisoft Montreal. Eric spoke of the methods used by Ubisoft’s team on the development of Far Cry 4, released in November 2014. Eric’s talk covered the diverse set of tools used when testing, which included eye tracking, skin sensors to measure sweat levels and register excitement, as well as telemetry in the form of tracking in-game actions and locating them on the 3-dimensional gaming map, and time stamping videos to observe those actions. Using Tableau software to sync these diverse sources of data, he showed numerous examples of how these tools helped make usability issues easy to identify and then report back to designers in the form of bite-sized videos. The multiple layers of data can be a lot to handle, but with intelligent use of software they can enrich the analysis of the player experience. His lesson can be applied across industry as well. Although we may not have the ability to perform eye tracking and other more costly methods with every study, we can use behavioral data captured from users as well as analytics data to help get a deeper picture of how users are interacting with the site. The more sources we can pull from and sync up, the more likely we’ll be able to identify key issues when testing.
The video game industry has grown extraordinarily in the past decade, and is now a $20 billion dollar industry in the United States alone. Learning from usability professionals in this field was a great experience, and it was especially valuable seeing how they attack usability problems specific to their field. I highly recommend the GUR summit to all usability professionals across all industries. There’s no doubt that we can all learn something from our colleagues in usability, as long as we take some time to step back and connect.