Gathering feedback is notoriously difficult. Most game developers gather feedback after a game's launch through completely automated systems. Devs then test their decisions through AB tests. AB tests however sort of rely on users being unaware of the tests happening, which is impossible here.
The game does have a feedback system, however it's not really made in a way which allows people to review how a change affected the various gathered parameters, additionally the parameters which are gathered mostly fall into the "fun things to read about" category, not the actually useful, but way less fun, parameters such as user retention, focus, engagement, session length, activity during sessions, action ordering, etc. + mapping how all these parameters change over time, over player age (how long they've been with TGS), over the course of an individual play session, etc. Lastly there is the extra question of whether players are actually willing to be tracked for feedback reasons, knowing that the gathered information would help coders see if the changes they do actually had their desired effect.
I can already hear your cynicism about how coders would not look at data. It's not necessary that they do, instead this could fall into the domain of an analyst, or anonymized data could be available to the public for scrutiny. If code leads impose expectations on coders that content which fails to produce the desired effect will get reverted, then this might lead to a gradual change in coder culture. It won't be instant though.
The system on github works fine. The 24 hour review and comment process is generally a good thing, that's how most development (game or otherwise) companies operate. It would be nice if "I don't like this, close" happened less frequently, but whatever, it's volunteer work, imperfection will happen. I however don't think the 24 hour review period is a good enough place for player input on changes. The period is too short for enough players to see the change ahead of its addition to the servers. I also don't think player reviews before the change goes live are necessarily a good thing. Not because the players reviews are bad, just because it would slow down updates a lot.
We also have the forums. Coders can post feedback requests and players can post topics giving feedback on various things. There are however two issues here: Many players seem to jump to sensationalism and absurdity in many of their replies. Sensationalism is when players write an over the top negative reply about some minor, inconsequential change, stating about how it is the worst thing ever, usually in more vivid language. Absurdity is when they cite counter-examples to prove their reasoning, which are extremely unlikely, rare or inconsequential. On the coder side, reading and replying to forum feedback is difficult, because it requires a lot of time to get through it all, it requires manual collation of how many people like it and how many don't, and it requires the coder to make assumptions about how many people are in the silent majority for and how many against the change. In other words, the feedback forums are completely useless to coders, except when they provide very detailed and thought out reasoning for the poster's opinion. There are ways of encouraging that, however it requires much closer forum moderation and many people would perceive it as censorship (or it could actually devolve into censorship if not implemented correctly). Examples include mandatory post forms, minimum or maximum word counts, minimum quality standards for posts to not get deleted, etc.
Ingame polls are also a thing. Unfortunately, however, these are difficult to use and would need to be used much more. These are much faster to respond to for players, allowing much more of them to state their opinion. They however do not allow players to post more detail about their opinion, they can only choose from one of the pre-defined answers. This does however allow coders to get a very quick tally of how a change is perceived.
One more thing which has already been tried, and didn't really work, is a dedicated beta server. This server would run the latest code and have daily code updates, with the other servers only getting updated once per fortnight or month (unless critical bugfixes require an out of cycle update). While this is common with indie studios, it may not work well here, simply because there aren't enough players to sustain it, meaning it would either get all the population or none of it.
tl;dr:
So with all that in mind, what would you say is the best way for coders to gather feedback? Think of how well each person can express their opinion's nuances, whether they can suggest improvements, how easy it is to respond to, and also how quickly a coder can aggregate the feedback.
Suggestions explained above are: AB tests, per-player feedback gathering (aka tracking), github reviews before changes go live, feedback forums, ingame polls and a beta server. More suggestions are always welcome.
What is a good way to gather feedback?
-
- Joined: Tue Aug 25, 2015 12:59 pm
- Byond Username: LiamLime
What is a good way to gather feedback?
Last edited by LiamLime on Sun Sep 06, 2015 12:28 pm, edited 1 time in total.
The bureaucracy is expanding to meet the needs of the expanding bureaucracy.
-
- Joined: Sat Apr 19, 2014 11:23 am
- Byond Username: Miauw62
Re: What is a good way to gather feedback?
polls are, imo, not really very good.
they tell you if players like or dislike something, but they don't tell you what people like or dislike about that thing, which is very, very important.
sure if nobody likes a thing after a few weeks it should be removed, but learning why it should be removed is still useful, and in many cases players can be divided on a feature and we need more detailed feedback.
they tell you if players like or dislike something, but they don't tell you what people like or dislike about that thing, which is very, very important.
sure if nobody likes a thing after a few weeks it should be removed, but learning why it should be removed is still useful, and in many cases players can be divided on a feature and we need more detailed feedback.
<wb> For one, the spaghetti is killing me. It's everywhere in food code, and makes it harder to clean those up.
<Tobba> I stared into BYOND and it farted
- Aranclanos
- Joined: Tue Apr 15, 2014 4:55 pm
- Byond Username: Aranclanos
Re: What is a good way to gather feedback?
why do you make a new account to post this
to don't derail too much, I like stats that I never have. All the time people talk about their feelings on balance issues, which is fine too, but the lack of real stats is exhausting. Things like percentages of victories across the different types of antags or even the reasons of why the shuttle is called, round durations (that's a bit harder to measure though). Stuff like that.
to don't derail too much, I like stats that I never have. All the time people talk about their feelings on balance issues, which is fine too, but the lack of real stats is exhausting. Things like percentages of victories across the different types of antags or even the reasons of why the shuttle is called, round durations (that's a bit harder to measure though). Stuff like that.
I accept donations, click here
- invisty
- Joined: Tue May 26, 2015 12:02 am
- Byond Username: Invisty
Re: What is a good way to gather feedback?
Where's the option for "NAAAAAAH I CANT HEAR YOU", *merges pr* ?
Mime: Depresso
-
- Joined: Tue Aug 25, 2015 12:59 pm
- Byond Username: LiamLime
Re: What is a good way to gather feedback?
Some of that does exist though.Aranclanos wrote:why do you make a new account to post this
to don't derail too much, I like stats that I never have. All the time people talk about their feelings on balance issues, which is fine too, but the lack of real stats is exhausting. Things like percentages of victories across the different types of antags or even the reasons of why the shuttle is called, round durations (that's a bit harder to measure though). Stuff like that.
For per round stuff you have (example for Traitor): http://ss13.eu/tgdb/tg/latest_stats.html#mode_traitor
That has the per-antagonist type success rate as well as how well or poorly each antagonist did on individual objectives. Round durations are also below
For global antag success rates (irrespective of round type or objectives) you have this: http://ss13.eu/tgdb/tg/latest_stats.html#antag_success
For global antag objective stats (irrespective of round type) you have this: http://ss13.eu/tgdb/tg/latest_stats.html#antagobjetive
For survival and escape rates, you have this and the graphs after it: http://ss13.eu/tgdb/tg/latest_stats.html#survivalrate
And for round duration without respect to round type, there's this: http://ss13.eu/tgdb/tg/latest_stats.html#rounddur
I however wouldn't call any of that extremely important data for decision making. Antagonists are not the majority of the player base during a round and focusing exclusively on them would be wrong. I also don't think data about shuttle call reasons is gathered.
The bureaucracy is expanding to meet the needs of the expanding bureaucracy.
- Not-Dorsidarf
- Joined: Fri Apr 18, 2014 4:14 pm
- Byond Username: Dorsidwarf
- Location: We're all going on an, admin holiday
Re: What is a good way to gather feedback?
A reminder that it's very, very rare for features to be merged after 24 hours. Almost all, even minor content ones, are made to wait something like a week.
kieth4 wrote: infrequently shitting yourself is fine imo
There is a lot of very bizarre nonsense being talked on this forum. I shall now remain silent and logoff until my points are vindicated.
Player who complainted over being killed for looting cap office wrote: ↑Sun Jul 30, 2023 1:33 am Hey there, I'm Virescent, the super evil person who made the stupid appeal and didn't think it through enough. Just came here to say: screech, retards. Screech and writhe like the worms you are. Your pathetic little cries will keep echoing around for a while before quietting down. There is one great outcome from this: I rised up the blood pressure of some of you shitheads and lowered your lifespan. I'm honestly tempted to do this more often just to see you screech and writhe more, but that wouldn't be cool of me. So come on haters, show me some more of your high blood pressure please.
Who is online
Users browsing this forum: No registered users