How Games Benefit from UX Testing

22 August 2018, by Cara Winterbottom

The success of any game is highly dependent on how fun, engaging, challenging and balanced it is. Regular playtesting during development can help you to maximize these qualities. The tests must be carefully designed to find where things go wrong and how they can be improved. Here’s why user experience (UX) testing metrics are highly beneficial for this.

Cara_UX_Article_image1

I’ve recently been the UX specialist for the development of an educational game. Geomax teaches geometry concepts to Grade 7 and 8 learners. Players collect objects, complete puzzles and avoid enemies while racing to the finish of each level.

We tested the game throughout to ensure that it was fun to play, engaging enough to be replayable, challenging enough to be interesting and not frustrating to learn. Designing new user tests for almost every sprint, I found that usability test metrics such as those defined by UX-giants the Nielsen Norman group applied well to game testing:

Learnability,
Error recovery,
Efficiency,
Satisfaction and
Memorability.

Let’s look at each of these in more detail.

Learnability of Games

In UX, learnability addresses how easily users discover the features of a product and learn how to use them. In a game, players should be able to discover and use the mechanics and control interfaces fairly easily and, of course, have fun while doing so.

It’s important that the initial discovery is fun and engaging, as it’s a player’s first experience of a game. This is where they decide whether it is worth playing.

For Geomax, we started testing the game with learners as early as possible to make sure that we got this right:

Before we had levels, we tested mechanics in standalone demos. For example, for avatar movement, we tested different virtual joysticks to find out the most natural and learnable one.
Initial level tests had no visual design or control interfaces like menus. Instead, we focussed on learners discovering how to play the game. We gave them no instructions or help until they showed high levels of frustration and were on the point of giving up. We refined the game, tested again and eventually added tutorials for initial interactions. We then tested these tutorials, observing players to make sure they noticed them, learned from them and had fun with them.
Where we introduced mechanics or extended existing mechanics in later levels, we applied the same process of refinement to make sure that learners understood the mechanics and how they could increase player expertise.

Testing learnability rigorously resulted in a game that players could pick up easily enough, had fun learning and that challenged players to think strategically.

Tip: Test mechanics and gameplay discoverability as early as possible. It is likely you will need a few iterations to get it right, so players enjoy the challenge and don’t find it too easy or difficult.

Common pitfall: If you test in groups, make sure the players can’t learn how to play from each other. While the social game is important, players will probably discover the game for the first time without the help of their friends. You must simulate this in your testing environment to really find out how learnable your game is.

Error recovery

In UX, error recovery refers to how many and how severe user errors are as well as how difficult they are to recover from. For example, navigating to the wrong page on a website is usually a minor error with easy recovery through using the Back button. In games, players learn by making errors. Examples from Geomax include forgetting to pick up items they need later or moving too close to enemy objects so they are attacked and lose items they have collected.

In order for the players to keep their interest in the game, the consequences shouldn’t be too damaging. They must be able to work out how to recover and avoid the error on their own.

While learners played Geomax, we checked whether errors resulted in good or bad frustration:

Good frustration is important in many games. Obstacles create challenges and keep players interested. When players groan or swear when playing, this is often a sign of good frustration.
Bad frustration occurs when players have to do too much mundane work to recover or can’t recover. They lose interest, stop having fun and consequently stop playing the game. Frustration also becomes a bad thing when it happens too often or for too long. We observed this when players struggled over the same mechanic for a long time compared to others, when they pushed away from the game, and when they said things like, “This is impossible!”

Whenever we saw bad frustration, we knew we needed to adjust the environment or interactions. For example, simplify or space out enemies, or shorten the path to lost items. This resulted in a game that players would not abandon because it was too grinding or difficult.

Tip: If your levels are short enough, players don’t have to be able to recover from errors. They should just be able to see how to avoid them when they replay the level.

Common pitfall: Don’t remove all frustration so the game becomes too easy. Unlike with other products, it is important that there is some challenge in a game to keep players interested!

Efficiency

In UX, efficiency refers to how fast and accurately users can perform tasks with an interface once they know it. The point of games is of course not to complete them as efficiently as possible.

However, players must be able to improve and feel that their skills are progressing so they can beat the game or each other if they play hard enough.

We tested for Geomax’ efficiency per mechanic and overall:

For each mechanic, we focussed on how player usage changed as the game progressed. If players still struggled with a particular element after repeated play, we knew we should adjust it. For example, we spent a lot of time fine-tuning the ease and sensitivity of avatar movement because learners repeatedly struggled with it. They felt it was too laggy and slow to accelerate or too fast and out-of-control. We refined this until we struck a balance where players felt they could master the movement style.
In order to test overall efficiency, we looked at the balance of the game. We asked the same learners to play in repeated sessions, with increasingly complex and difficult levels. We noted how their play and their attitudes to the game changed. For example, if players showed signs of boredom, like yawning or lack of enthusiasm about playing the next level, we knew the game needed more challenge, difficulty or excitement. Because Geomax is a competitive racing game, it has a medal system and leaderboard. Once these were in place, we observed how players’ level timings improved on replay, which was a good sign of increasing expertise.

This ensured that we created a balanced game that remained compelling and engaging as players improved and gave them pleasure at developing their skills.

Tip: Remember to focus on both individual elements and the overall structure of the game. Game environments, objects and mechanics interact during play. This changes how each is perceived by players and can impact discoverability and difficulty.

Common pitfall: Do not be rigid about the structure and organisation of levels and mechanics. Balancing a game is a difficult, delicate and ongoing activity. Even towards the end of development, you may want to move or even remove levels, elements and mechanics of the game to create a well-balanced game.

Cara_UX_Article_image2

Satisfaction

Satisfaction -unsurprisingly- refers to how pleased users are with their experience using a product. This is tied into whether they can achieve their aims easily and successfully. It also refers to the amount of delight and pleasure an interface gives the user. Games are all about fun and challenge, so satisfaction is very important.

In addition to compelling gameplay, social engagement and appropriate rewards, the narrative and game art should be inspiring and attractive.

We tested for satisfaction in various ways at different points of development:

For gameplay satisfaction, we asked learners what part of the game they liked most and least, and why. We also asked them what they thought about various game elements, and paid attention to how positive the responses were.
To test narrative and aesthetic effect, we asked for player responses to different styles of artwork, avatars, obstacles and other objects. Although Geomax does not have an explicit narrative, we asked learners to make up stories about the game so that we could judge how inspiring the game look and feel was.
To judge social engagement, we observed how learners interacted with each other while playing. Once we had a medal system and leaderboard, we could observe player reactions to their relative positions on the leaderboard.

As well as asking about game details, we observed players for signs of enjoyment, for example:

Cries of pleasure or surprise,
Laughter,
Bantering and competition with other players and
Showing desire to play more than is required for the test.

As designers, we then focussed on what worked for players and enhanced those elements. Where players showed less satisfaction with the way a particular element or mechanic worked, we fine-tuned it by changing its location and increasing or decreasing the challenge. We also looked at overall game balance and changed the way levels were ordered to improve it.

Tip: Just asking about satisfaction rarely yields good results, because test participants tend to want to please the tester and game players are no different. Focus more on observing player reactions and behaviour than on asking about satisfaction. If you do ask, make sure you get detailed responses and reasons rather than yes or no answers. You can use the details to judge how satisfied players really are.

Common pitfall: Do not ask testers whether they would buy this game when it is released and how much they would pay for it. The results will not be accurate. Players may be excited about the game now, because they have been playing it with friends, and you have likely paid them to do so. They cannot know how they will feel in the future, when they must pay and play in their own time.

Memorability

Memorability is about how easy it is to return to a piece of software after a while and still know how to use it. In games, the situation is a little more complicated. Players should retain their expertise when they haven’t played for a short time, but they shouldn’t be able to jump into higher levels and play expertly without practice.

Players build up skills in games. If these are too easy to pick up, or disappear too quickly without ongoing practice, they may seem trivial or not worth the time spent building them up.

We tested Geomax with repeated groups of learners to ensure that the controls and mechanics were memorable. We observed how long it took them to reorient themselves and modified elements that prevented them from jumping back into the game easily.

We also worked constantly on the game balance. Higher levels needed to be complex and difficult enough that players felt challenged when they returned to the game after a time gap. Behaviour that we observed to support this included:

Players asking to go back and replay earlier levels so they could practice before attempting later levels.
Players noting that their level scores were lower than they had achieved in previous sessions, and replaying those levels to try and improve them.

This ensured that after gaps of two weeks to a month, players would be interested in picking up the game again, and were encouraged rather than discouraged from playing after an absence.

Tip: Game interface elements like movement controls, buttons and menus should not hinder play in any way. It is very important that these are intuitive and easy to remember, so that players can just focus on their game expertise, rather than trying to remember how to Pause a game, for instance.

Useful resources

Strip-1

Cara Winterbottom is a UX consultant and author who has trained extensively in both Psychology and Computing and has a passion for both. The skills and knowledge that she has learned in these areas are powerful tools that she brings to bear on her user experience and software design work.