Industry insights: A Jury of Peers: How Independent Game Developers Depend on Feedback

A Jury of Peers: How Independent Game Developers Depend on Feedback

By Anne Gonschorek on August 31, 2017

Free Lives is an independent South African video game developer based in Cape Town. The studio is best known for Broforce, a game that sold over 1 million copies while still in development and, by May 2016, had made almost $3 million. We are chatting to Creative Director Evan Greenwood to find out how Free Lives evaluate their prototypes.

alt

In Evan’s own words, Broforce is a loving parody of the portrayal of American military interventionism in cinema: American action heroes of the 80's and 90's are fighting terrorists in Vietnam with Satan himself as the ultimate enemy. This seems to have clicked with the critics: Broforce gained a 78% score from PC Gamer and 8.8 out of 10 from IGN.

The South African gaming industry

While Evan prefers the comparatively comfortable lifestyle South Africa offers an independent gaming developer, the industry is relatively small and isolated from the rest of the world. Only about 31 companies responded to the last survey by IESA (Interactive Entertainment South Africa Industry), the local industry body. “There aren't a lot of veterans to learn from,” he says. “It's also quite hard to regularly attend overseas conferences, to get feedback and make contact with potential business partners.”

alt

The revenue growth of South Africa’s game development industry spiked from R30 million in 2014 to R100 million in 2016. Although this is significant growth for a small industry, making indie games is not for the fainthearted.

“The way the market works is this: The top 10% most successful games will be making 90% of the money and the bottom 50% make no money.”

While South Africa has a lot of creative talent in the form of animators, artists and developers, the talent pool of those interested in making games is still quite small - and the market is competitive.

The challenges of the gaming industry

“You are competing with all these other industries, like the animation and programming scene,” Evan explains. Game development offers a lot of excitement and creative fulfillment. But, at least in the indie scene, it also comes with a lot of risks and question marks, such as the uncertain success of a game and the potentially long time- and emotional investment. Broforce, for example, took three and a half years to complete.

alt

This does not only make it difficult to schedule but is also hard on the psyche:

“Humans weren’t built for working on abstract things for three and a half years. We’re built to work on something that would give us food or shelter tonight.”

The abstract nature of game development makes it difficult to see and feel progress being made. According to Evan, a lot of game developers experience burnout. Despite having invested large amounts of time to hone their skills, some can later not find the motivation to work on games anymore. “That’s why a bad game can kill a studio, can kill people - not literally, but for the gaming industry.”

While working among friends helps to avoid this, Evan also swears on staying in a constant feedback loop with the audience - not just to ensure the game stays on track but also for the motivation of his team.

Testing through audience feedback

In a movie, one scene changes to the next in the same sequence, no matter how many times it is being watched. That is why the experience will be more or less the same with every audience. But this is not the case with games. While the functionality of a game can later be ensured through professional testers, the gaming experience largely depends on who is playing, especially in interactive games.

This is one of the reasons it is so hard for game developers to test their own games and foresee the experience their audience is going to have: “When you are developing a game, you learn how to do everything. You learn how to play it perfectly. You have full knowledge of how it’s meant to work. That makes you very much a spoiled test subject,” Evan explains. That is why the majority of game testing at Free Lives and studios like it happens through user-generated feedback, right from the start.

Showcasing a small prototype

Broforce, for example, was created at a game jam, a kind of hackathon that happens over a weekend. Starting with a blank Unity project, Evan and his team began by making little characters move along. “Immediately, we were getting some people from the jam playing it, commenting on it and talking about their experiences,” Evan explains.

alt

This process is quite common: Developers bring their idea to the jams and play each other’s prototypes. Game developers understand the need for testing and feedback in order to shape future decisions: which parts to remove or add, where to double down. Because other game developers are not only most accessible but also most sensitive to the requirements of quality games, they provide the first litmus test of whether the game idea is viable.

Getting public feedback

“If we get no feedback or mostly negative comments, we would probably drop a game,” Evan says. Even in the early stages, external input is indispensable. In the case of Broforce, people could already play the first raw bone prototype 72 hours after the game jam online.

Many of the decisions about what the game would later become were shaped by what early playtesters found exciting. While players won't be able to give feedback on how to fix or improve the game, they will tell the developers where they were enjoying themselves the most and the least. “The cast of action heroes was generating the most chatter, so we kept adding new heroes and detail to existing ones: We went from 4 heroes each with 13 animation frames to 30 heroes with 3 unique attacks and hundreds of animations frames each,” Evan says.

alt

This feedback also keeps team member’s expectations realistic - an important factor for projects that can potentially last over two years. “A lot of it is about intuition. You’re trying to fill in gaps but you can also check some of the analytics.” As a prototype goes through iterations and improvements, the team keeps releasing new versions for their audiences - first for free and later for a reduced price while the game is still in development: “With Broforce, after about two years of work, we gave gamers early access - it was still in development but because we had been playing and refining the prototype, we had about an hour’s worth of game play,” Evan explains.

Understanding Feedback

Seeing how many people successively play the game and whether it is a growing audience already serves as a first indicator of a game’s later success. In the beginning, however, the assessment is a lot more delicate. According to Evan, the trick is to understand the difference between a weak positive result and a true positive result: One is a game that people enjoy when they play it but very few people will actually end up playing and the other is a game that will generate actual interest and further sales. “People have to be so excited they want to tell their friends about it,” he says.

alt

When the idea for Broforce was born, the Free Lives team was already half a year into the development of another project: Slice Fight, a zombie slicing game for mobile platforms. Despite the time they had already invested, Broforce’s overwhelmingly excited reception had them kill the other game: Four people had been improving a prototype for six months and actually received positive reactions, but not the right kind:

“If people say: ‘This is cool’, you should probably stop working on it.”

Broforce, in contrast, had people asking for the final game release. They wanted to play the full experience: “If people want to have an hour long conversation about it with you, that’s when it’s probably good,” he says.

Where to get feedback

Local forums

In order to get and keep up with the feedback on new prototype versions, game developers make use of local forums like makegamessa.com - the largest South African game developers forum, and game hosting platforms like itch.io and gamejolt.com. “In these forums, you are explicitly asking people to respond,” Evan says. But because it is quite difficult to extrapolate actionable data from comments on online forums, indie game developers also use other means to interact with their users.

Meetups and festivals

One of these means are meetups. MakeGamesSA, for example, has monthly meetups in both Cape Town and Johannesburg. A developer team can then bring their game - be it on a laptop or projected onto a screen - and let attendees play it before asking directed questions: “First, you want to get functionality questions out of the way, so that you can get deeper feedback about the experience,” Evan says. The team tends to focus on the most recent additions to the game, because these are the parts that the developer knows the least about.

Festivals are also great for getting feedback. Free Lives took Broforce to a few overseas festivals like MineCon in Florida and PAX in Boston and Seattle. Setting up a booth, they could watch people play the game and both see how they played it as well as how they reacted to it.

Analysing video streaming

Another way to get a lot of feedback are Youtube and Twitch recordings of gamers playing new game versions. “Throughout development, the team's tendency was to make Broforce more and more challenging, because the Free Lives team grew more masterful at playing the game,” Evan says. But once they saw how early YouTubers were playing Broforce, they noticed some of their frustrations. “At one point, a pair of very popular YouTubers spent 45 minutes trying to beat a single Broforce level, and ultimately failed and gave up,” Evan recalls. That’s when the team knew they had to start scaling back the difficulty.

Incorporating feedback

In addition to understanding what changes need to be made, each feedback loop is also an opportunity to find out whether improving the game further is worthwhile. That is why the team constantly needs to ask themselves a couple of questions:

  • How much improvement is still required for the game to be commercially viable?
  • Will the game be commercially viable before the costs exceed the predicted sales revenue?
  • How does this game compare to other potential game projects the team could build?

alt

When the Free Lives team had reached the finish line with Broforce, they put the game on Steam Greenlight for final validation. On this platform, now phased out and replaced by Steam Direct, the audience votes which games are greenlit for distribution on the Steam marketplace.

“In Broforce’s case, we launched the game after the people playing the pre-release said that they’d happily pay money for the experience. I think that’s a pretty good yardstick.”

Cat eyes@2x

Subscribe to our blog

Don’t miss out on cool content. Every week we add new content to our blog, subscribe now.