This is going to be a long one, so apologies in advance.
One question that all game developers will come across at some point during their career is “should I take that feedback on board?”. Someone, somewhere, gave you feedback on your game and now the issue is: why should you listen to anyone regarding your game? Surely you know better. And even if that suggestion or feedback is a good idea, then people will complain that “the game is dumbed down”, “this is not what I hoped it would be”, “the game caters to whiners, noobs, QQers and casuals” etc. Go to any official game forum and you will find a plethora of people giving feedback to game devs on how to make the game better, and a plethora of people telling the game devs that if they were to take that suggestion on board, they would start a riot. This post will look at the various pros and cons of taking feedback on board during and after game development, who the sources of feedback are and what answers those sources can give regarding the development of a game. Hopefully I will help clarify the differences between user research and market research (in this case, community feedback). I will only touch upon player/customer feedback, as I cannot comment on managerial or intra-team feedback.
Keep in mind that none of the following methods can tell you how to make your game. They can only help guide your design choices.
No matter how early or late into development, a game will have followers. Players, customers, users, adopters. However they are called, they form a game’s community. Keep in mind that I will focus on the vocal online community, but that is only a fraction of a game’s playerbase. A person who buys a game, plays it, but never goes on a forum or the company’s website is still a customer with an opinion that is just as important as everyone else’s.
-What you can get out of community feedback:
Communities are a great way to get answers to questions such as “should artifacts in our game be obtainable through progressive grinding or found by random luck?”. By creating a poll, for example, you can get a good representative sample of your entire customer base to reply and see where players stand on that point. This does not mean that you get an answer to which option is a better design choice (both might be terrible ideas) but you can get quick and tangible feedback on what the community thinks of two clear-cut choices that are easily understood by everyone. If 70% of the poll’s participants vote in favour of the random luck option, then you will know where most players stand. Community feedback shines when it comes to a problem that a majority of people encounter: when Planetside 2, by Sony Online Entertainment, was first released, a very vocal majority of players came together on the forums to complain about the enormous amount of time required to grind through experience points gain in order to acquire even the simplest of upgrades, such as a scope for one gun. Following the uproar, the numbers were tweaked so that the necessary grind to unlock various features of the game was lowered. As this happened through the Beta, the numbers were ready for the actual release, and no further complains were noted. This would have been very hard to spot through usability testing, for example, as this required a large number of people to get a feel for the game over a long period of time. Finally, community feedback allows you to help define your target audience. Usually the answer to “who will play your game?” is “everyone of course!”. But in actuality, not everyone plays every game. Seeing who expresses themselves and about what can help developers find out who cares about their game and what expectations those people have. Then they can start catering to them or ignore the audience that is of no interest to them.
-What you can not get out of community feedback:
Whether through a poll or a survey, the answer to why an option is picked over another will remain unanswered. You will not be able to understand whether people are in favour of an option because of the context of the setting in which that option appears, or whether it really works for your entire game in particular. Sometimes, players have a particular schema in their head due to competitor games having ingrained a particular way of doing things so deeply into their minds that it becomes hard for them to imagine a game without that particular option. Furthermore, community feedback is a very strong reminder of why market research is so tough on designers: you will never be able to please everyone. The more voices are brought into the equation, the higher the number of displeased vocal people becomes. That is another weakness of the market research approach to community feedback – the fact that negative attitudes are very bad predictors of behaviour. Put simply, just because someone says they don’t enjoy something does absolutely not mean they will not do it. People might say that they dislike a particular feature or design choice but they might never have thought so strongly about it if they had never been asked. More often than not, people that complain will still use a product and be mildly annoyed every now and again. I am not trying to make a case for “try and get away with as much as possible” but rather a reminder that people don’t know what they want. So when asked, the chances that they ask for something specific are slim, while the chances for them to ask for something “different than this” are high. In the end, when doing market research with large numbers, one gets is a large number of anonymous, emotionless answers. They can be constructive and detailed, but it is impossible to understand what really triggered and motivated that feedback for the particular player that gave it.
From the very first day of development, from concept ideas generation to the final build ready for release, user research aims to identify gameplay, usability and generic issues with a game. It cannot make a game fun, but it can help weed out bad design choices that might hinder that fun for all the wrong reasons. That is true for any project, big or small, and for any developer team, gigantic or tiny.
-What you can get out of user research:
By getting a few select individuals to play your game, or give feedback and their thoughts on an early prototype, or even point at a wall and tell you what their preferred colour palette for a dark fantasy setting is, you can get quick and constructive feedback on pretty much any aspect of your game. The enormous advantage is, you can then directly ask those people for the why and how they perceive that aspect of the game the way they do. It can range from something simple to something very complicated, but almost 90% of the time it will be something that the designers and developers of the game cannot see or predict because they are too close to the game they are creating. User research gives a face and emotions to players’ opinions, and it opens the opportunity for debate on things that the game’s designers consider a given. When you have spent months unending on a piece of software, you know the ins and outs of it. And you cannot possibly imagine how someone cannot see things the way you do. That someone, however, does not have the amount of experience that you have in this particular instance. And more importantly, that someone is the one who will end up buying the game. The designer’s view and idea of a game can be shared by players who are equally interested in it, but the way the idea is executed can be miles apart due to the gap between the designer’s mental model and the player’s mental model. User research can very easily and quickly answer the questions: “How can they possible not get this? How can everyone get stuck on this level? Why is everyone dying here? Why is no-one using that amazing tool we spent months designing?” and the all-time favourite “How can they not see that? It’s so obvious, it’s right in front of them!”. Furthermore, user research usually is best combined with benchmark analytics of your community’s data, or even informed by it. If you see a large behavioural trend of what people are doing in your game, user research can help enlighten why they are doing that.
-What you can not get out of user research:
User research can help you with hard-to-answer questions and look at your entire game, from the small details to the bigger picture. The hard part of user research however, is that you have to ask the right questions in order to receive the answers you need. It is often the case that developers do not know what they want out of research, and sadly they perceive it as a step done at the very end right before release just to confirm that every decision they’ve taken is the right one. User research can not tell you how to make your game fun, or how to design a level so that it’s engaging. It will tell you that a particular part of your game that you thought was paramount to it being enjoyable is actually being skipped and thus people don’t really engage with it in the way you would have expected, but it will not tell you how to create an element that generates fun or engagement. Finally, user research is not the best approach to evaluate market trends and general consumer expectations regarding a genre (e.g. to see whether a demand for a game likes yours is there), but can only highlight user expectations regarding your game based on past experiences.
Different sources bring different answers to a variety of questions. Use your community to help you shape questions that can then be answered through user research. Interpretation of data, no matter how large or small your sample is, is the alpha and the omega to developing your game your way through informed decisions. If people complain that your game is too hard and that is exactly what you wanted, then you have succeeded. Your game will be fun for a smaller audience, but it is the target audience that you chose to accept feedback from.
The point is not to ask people how to make your game. The point is to ask people what they think of how you make your game.
I highly recommend reading two articles on the use of user feedback during game development, one talking to various professionals in the industry and the other giving a concrete example for a small indie studio.
Cover image © Guild Wars 2 Concept Art