Why is player feedback so primitive?

[Oldman Robustin]

Seriously, if someone tasked me to come up with the least efficient system of gathering feedback I would just screencap github comments, feedback threads, and OOC and submit it as my thesis for the worst feedback system ever devised by a group that ostensibly values user feedback.

Even if we developed a process that did a better job of incorporating the github/forum/ooc comments, it would still do very little for developing useful information. The best thing we can do here is shout anecdotes and foist up shitty polls with obvious bias in the choices. Our proposals here are completely pointless fantasies that will never see the light of day unless a random coder swings by (and so few do) and decides to take up your cause. Even still, the process here is deeply flawed and basically limited to a few voices with no guarantee that any of them actually stand for what the average player thinks.

Polls are an obvious step, but I physically cringe at what I see - clearly biased language in the choices, straw polls that are literally just blank text fields with a massively open-ended question, and it's still constituted by a fraction of the playerbase and by many people who scarcely play the game.

So here's a simple proposal. At the end of every round that a player actually participated in, i.e. had a position on the crew at some point, the round gets an end-of-round survey that asks some basic questions about a player's enjoyment of the round. It would ask the kind of questions that give us a robust analysis of how the crew feels about various roundtypes. Questions like:

"What role were you assigned to?"
"Check which, if any, of the antagonist roles you were given this round"
"Check which of the following describes your arrival on the station" (Roundstart, early-latejoiners, etc.)
"When compared with your average round here, describe your enjoyment of the previous round" (Well-Below/Below/Avg/Above/Well-Above)
"Did your character survive the round? Escape the station?"
"How long have you been playing SS13 on this server?"

At the end of every round, this survey would open and players could finish it while waiting for the next one to start. There could be additional incentives (One out of every X responses gets an admin wish/steam gift/whatever) for completing it. Additional data could be attached as well like the survey response rate, braindead %, the usual antag success/failure stats, and other data possibly entered manually in a special admin survey (e.g. "Any admin events this round?" "Number of bans handed out").

Of course this wouldn't be a perfect system, there would still be a slew of biases and predispositions that would skew the data... but within a matter of days we would start getting a MUCH clearer picture of where the game is succeeding and where it's failing. A careful look at the data could also reveal other patterns:

When does security have the most fun?
What correlation is there between death and enjoyment across various game-types?
Is there a correlation between time played as captain/hos/heads and the crew's overall round enjoyment?

It goes on and on. The use of data to drive changes would be a nobel-prize winning breakthrough compared to the system we have here. Plus the end results would be fun to look at.... what IS the relationship between security fun and clown fun?

[Not-Dorsidarf]

While It's too late for me to really address the ideas behind this, most of your questions (survived, was antag/etc) can be automatically input by the game to avoid shitbreads saying they were nuke wizardlings who hated the round (after Rev)

[Oldman Robustin]

While It's too late for me to really address the ideas behind this, most of your questions (survived, was antag/etc) can be automatically input by the game to avoid shitbreads saying they were nuke wizardlings who hated the round (after Rev)

Yea, I know some of this stuff is tracked already... not sure how easily that information can be tied to the BYOND key of people responding to the survey though... of course more automation of this would only be a good thing. Ideally players could just "rate their round" and the system would note how long the round was, antag types, what role the respondent played, how active the respondent was during the round, etc.

[iamgoofball]

I'm all for more stat tracking and polling of players. This is a very good idea, and we should pester MSO to implement it.

[Douk]

I think that, when it comes to polling things regarding game balance and development, that the polls available in-game are best to be expanded on due to it limiting input from people who don't actually play the game as well as greatly expands the opportunities for input for those who don't normally hang around the forums. The biggest problem with the in-game polls as it currently stands is the fact that it's something of an easily-forgettable/ignorable footnote that is only seen at roundstart. Thus, the number of people who actually participate in these polls may be significantly limited in comparison to the number of people actually playing the server. When acquiring user data like that, we should strive to get EVERYONE involved.

Thus, I would suggest implementing some manner of compulsory poll window that perhaps opens upon round end (as Oldman suggested) or upon ghosting. In the poll, ensure that links are provided to some manner of designated discussion thread regarding the issue (for those who are unaware of the ongoing discussion), and perhaps designate a trusted non-developer to write the polls to try and avoid bias. In all polls, regardless of content, provide an option to delay voting for those who have not yet initiated with the issue and want some time to educate themselves before voting (poll window is disabled for the rest of the round and vote not yet counted), and abstain for those who don't feel strongly enough on an issue to give input (vote is recorded but considered null; disables the poll window until a new poll is available).

The community we've fostered is larger than the usual cast of posters here on the forums. There is a large, silent mass of people who are server regulars who's voices too often go unheard. I know this server is not a democracy in terms of development and direction, but it disgusts me when "popular support" is defined by the will of what is likely a minority of the actual population.

[MisterPerson]

While It's too late for me to really address the ideas behind this, most of your questions (survived, was antag/etc) can be automatically input by the game to avoid shitbreads saying they were nuke wizardlings who hated the round (after Rev)

Yea, I know some of this stuff is tracked already... not sure how easily that information can be tied to the BYOND key of people responding to the survey though... of course more automation of this would only be a good thing. Ideally players could just "rate their round" and the system would note how long the round was, antag types, what role the respondent played, how active the respondent was during the round, etc.

Yeah that'd be fairly easy. A lot of that stuff is already tracked, like antag types, antag success rates, round length, etc. I don't think any of it is tied to a key because it's all anonymous. Plus i don't know if our current stats are being hosted anywhere, but they are being gathered. Plus the system makes it really simple to add new stats, which is nice.

Anyone could add this, not just MSO. Just start gathering the stats and nag MSO about making a better stat page to display the stats once they're gathered.

[duncathan]

This is a fantastic idea, Robustin. The surveys should be anonymous unless indicated otherwise by the survey-ee. There should be a couple different surveys, I'd think. One could be a simple poll; "On a scale of 1 to 5, how much did you enjoy your round?" Another could be a little more extensive:
"On a scale of 1 to 5, how much did you enjoy the following?"
- Your job
- The antagonists
- Any random events (would be shown if any were triggered)
- Any admin events (would be shown if toggled by an admin)
- The round as a whole
- Anything else you guys think of

Finally, there could be a full on survey:
- Everything from the above poll
- "Select a change made recently and give your feedback on it." (would have a dropdown menu of stuff from the changelog? probably a better way of doing this)
- "Describe your round in a couple of sentences."
- "What would you have done differently in the past round given the knowledge you now have of it?"
- Anything else

The survey would also send with it: player's antag status, player's job, whether player latejoined (and when), survival status, player age, connection count. Maybe drop the last two if that's too much a breach of privacy.

Doing this would give us hard data, and from data conclusions can be drawn.

[Oldman Robustin]

The community we've fostered is larger than the usual cast of posters here on the forums. There is a large, silent mass of people who are server regulars who's voices too often go unheard. I know this server is not a democracy in terms of development and direction, but it disgusts me when "popular support" is defined by the will of what is likely a minority of the actual population.

That was the sentiment behind my post too. There's no solution for people changing balance when their "fucking with code" to "playing the game" ratio is 20:1 and that '1' is spent playing botanist making giant weed plants and spouting dank memes where you'll get absolutely no perspective on how balance plays out. But even that problem is small compared to the fact that actually non-coder feedback is limited to like a half-dozen active posters here. There's absolutely no mechanism to gauge how the hundreds of non-forum players feel.

Which is really silly if you think about it, because compared to most game experiences we're uniquely well-suited to gathering player feedback in-game.

[Steelpoint]

I should note that its stated as fact by many coders that they don't usually browse the forums on a regular basis.

[Oldman Robustin]

This is one area where everyone isn't butting heads and there's virtually no downside to implementing this.

It would also be a salve to a lot of the other things I'm complaining about. Comments like "Traitor is seen as our best game mode", as an excuse to completely shut down any major traitor reforms when it's literally just one person's opinion... is not how a healthy system operates.

Going to keep pushing this until I see signs that there's an effort underway and not just the usual "oh that's nice" attitude from the people who can actually implement this.

[Ikarrus]

Traitor used to be the best game mode and for the longest time. Or at least the one people argued about the least.

But then etc etc happened and now we end up with a group of people just twiddling their thumbs IC or arguing over stupid bullshit OOC.

[Oldman Robustin]

Traitor used to be the best game mode and for the longest time. Or at least the one people argued about the least.

But then etc etc happened and now we end up with a group of people just twiddling their thumbs IC or arguing over stupid bullshit OOC.

It takes several months for opinion to catch up with reality. When you ask people "What do you think of traitor", they reflect on their entire traitor experience and would usually recognize that it was the stalwart reliable gametype compared with all the crazy imbalance that other rounds have endured.

Traitor has also seen almost no definitive changes, so its not like asking people about ling when there was a clearer split between newling/oldling and an understanding that thoughts about oldling were pretty much irrelevant to their opinion on newling. People will still reference their traitor rounds before flash had an AOE, before EMP kits didn't fully drain lasers, before/after/after-after helmet cams, etc., and all the other small changes that have resulted in a big problem.

Even with drastic changes like Newling, it took people... what... over a year to realize we had gone from a controversial but viable game-mode into a spider+LSD extended round?!

That's why testing for enjoyment on a per-round basis is much more important than "HEY KIDS DO YOU LIKE MAGIC?!" and other useless polls that beg to be gamed or subjected to overwhelming biases.

Also the fact that we talk several times a month about traitor being in the dumps on Feedback and Cheridan is acting like its a shining example of a gametype that we shant touch for we might taint its holiness is a sign that maybe some coders are out of touch and only hard data might get them to change their opinion. I mean we all talk about the lethality poll when discussing balance now, did Cheridan think that poll was a reflection that BLOB/WIZARD/NUKEOPS/REV wasn't lethal enough? FUCK NO, most of those rounds are a slaughterhouse. Meanwhile (at the time of the poll) we had traitor/traitorling/ling/DA making up 3/4ths of our games. Maybe, just MAYBE our opinion on lethality was a direct reflection of the fact that people felt like traitors and lings lacked the lethality necessary to create and sustain a conflict aboard the station... a REAL high-stakes conflict like "oh my god who keep murdering engineers" and not "Who keeps pooping spiders" and "Who keeps reading WGW through a voice changer?".

Err yea, data is good.

[Incomptinence]

If traitor is so sacrosanct why are we keeping the screwed scaling and not reverting to the older numbers from the golden era where people actually said traitor was the best mode frequently? I would say do away with large syndibombs entirely. TTVs are more robust for ops and more visible and easier to fight than macrobombs. The large syndibomb was instrumental to ruining traitor since suddenly our antag count was rationed like the bloody valves in toxins. The real kicker is it isn't even much good we ruined traitor for a mediocre more safe explosive.

[oranges]

If polls were easier to make people might actually run more of them

[Steelpoint]

What was the scaling for older traitor back in the 'good ole days'?

[Incoming]

What was the scaling for older traitor back in the 'good ole days'?

5~6

[Steelpoint]

That's for every five or six people there's one traitor correct?

Well that sounds like a good number to go with, that and with TC changes (if ever) that might go a long way in helping traitor.

[Ikarrus]

Well, I implemented the 2-tier scaling because at the time a high population usually meant security got increasingly too preoccupied dealing with the growing criminal element (greytide) to deal with any traitors.

I guess the recent crackdown on greytiding along with assistant caps really killed that. Kind of a shame, really.

[Ikarrus]

Here's the chart I made for the PR, dated September 15th 2014, nearly a year ago


The number of antagonists assigned in Traitor will be the lower of the two lines shown here.

X axis: Number of playing players
Y axis: Number of traitors

So the rough average sybil population of 50 playing players will have 6 traitors.

[Amelius]

Here's the chart I made for the PR, dated September 15th 2014, nearly a year ago


The number of antagonists assigned in Traitor will be the lower of the two lines shown here.

X axis: Number of playing players
Y axis: Number of traitors

So the rough average sybil population of 50 playing players will have 6 traitors.

That's WAY too low. That's almost half of what we had before (flat 1:5-1:6 ratio -> 1:10~ ratio at 60 people). No wonder traitor rounds now always feel like extended. Greytiding is rare nowadays, and even when it isn't, I'd, and probably most people, still would prefer a chaotic 12/60-man traitor round where things are actually constantly happening, with much lower security restrictions that allow sec to handle the few greytiders that still exit, without divine intervention, instead of extended-lite with 6/60 people being antagonists. I mean, you can't even sign up as an assistant sans roundstart, so you end up with very few, relatively speaking, assistants. You know there's a problem with a group of unallied, unidentifiable (to each other) antagonists who have to take on validhunter, crew, AI, and security alike, tend to be outnumbered by security alone, who not only are better armed, but also have some semblance of solidarity and teamwork.

It would go a LONG way toward buffing traitors and making the most-played gamemode more enjoyable again. Given that I've been hearing 'buff antag rates' for the past year or so, I'd venture a guess that it's a wanted change. Just do a trial of bringing back flat 1:5 traitorchan ratios if our gracious lord Cheridan finds it pleasureable. If that is found to be too high, go with 1:6 or 1:7.

I feel that a year~ ago was when I noticed that traitors started becoming pseudo-extended mid-round because most of the antags were caught early on. At first I didn't know exactly why that had happened, but I hazard a guess to say that progressively, through nerfs to traitors and buffs to security/QoL improvements for AI, traitors learned that going loud resulted in instant destruction and death, and so it became a progressively more rarely used strategy. Combine that with drastically reduced antag rates, meaning each antag round became more precious and there was less distraction for both security and the AI, secmaint and secbuffs (i.e. helmetcams) making there be no relatively safe area or murder for traitors, it encouraged stringing out your traitor round as long as possible (i.e., steal/kill your objective and sit around inconspicuously doing nothing, maybe acting like an asshole or doing some minor pranks that won't result in a permabrigging), only going loud on the shuttle or whatnot. I'm guessing this fully justified mentality is what caused our current predicament in totality.

[Ikarrus]

I'd still caution against simply increasing the number of traitors. We already have another mode that does this and it's reviled by many for being mindless chaos.

I'd much rather see people get over their fear of getting killed and give traitors some actual killing power. Making security and the AI less antag-proof would go a long way as well, but that also means sec and AI players need to suck it up some.

[Arete]

I'd still caution against simply increasing the number of traitors. We already have another mode that does this and it's reviled by many for being mindless chaos.

This is one of the big reasons I'm looking forward to datum antags. Injecting new antags into the round as old ones get caught means that you aren't making the round more boring every time you catch an antag, and there can be enough antags to keep things interesting without ever having so many at once that it becomes a random death bomb fest.

That said, just because there's a long term fix in the pipes doesn't mean that we shouldn't pursue a short term fix for the time being.

[Amelius]

I'd still caution against simply increasing the number of traitors. We already have another mode that does this and it's reviled by many for being mindless chaos.

I'd much rather see people get over their fear of getting killed and give traitors some actual killing power. Making security and the AI less antag-proof would go a long way as well, but that also means sec and AI players need to suck it up some.

But the thing is, as you said, a year ago we had that number of traitors. Do you not agree one person to antagonize 10 people is far too low, especially given that the robustness required to handle more than a singular person at a time is much greater than it is to, with allies and the AI overlord, to apprehend a singular suspect? The allied security force tends to be larger than the unallied traitors alone for christ sake (AND better-armed, the only part you propose to remedy, through some mysterious method), not taking into account the AI, validhunters, and so forth. Then you have traitors killing traitors since they tend to hang out in similar areas (maintenance), incentivized even because you can obtain more of that sweet traitor gear that will help you survive, and It becomes almost an unsurmountable obstacle. I'm telling you, as-is it's literally impossible if you aren't Oldman Robustin to antagonize the station for more than a short period. Traitors could gain lingchat and become an allied entity and they'd still get rolled, because they have neither the equipment nor the numbers required to do so.

Your rationale to add doublescaling for antag rates was because we DID have a greytiding problem in the past. We hardly do any more, so does it not make sense to remove that doublescaling? Also, we have far more sec players than in the past, because now rolls for antag are done before job roles, the effect of which is somewhat mitigated, only because playing sec is boring as hell playing much of the time since there's little antagonism.

Another point in favour is that you aren't proposing anything. Changing the antagonist rates is fast and easily revertable. Yet, you're doing the same thing everyone else has done for the past year that has made traitor the worst gamemode - passing the baton to something that sounds nice in the future, nothing concrete, while not being willing to entertain a possible solution. Cheri already shut down the 30 TC buff, which, while that wouldn't actually solve anything, it would help.

I mean, don't you agree that relying on rare robust folk to carry rounds, maybe one person out of the six, to be able to antagonize the entire station and act as the primary antagonist for the entire round, while everyone else gets caught within the first 20 minutes is a major reason why traitor has become what it has? Additional resources will enable better performance, but it won't make the unrobust robust, and hence won't actually fix part of the core problem - antagrates. Traitor was known as one of the best roundtypes a year or two ago, while DA was reviled, both with old antag rates. If you find reverting the former to what it was before the scaling was put would be a detestable change, then please, explain why and how people enjoyed traitor with those rates a year ago?

[Ikarrus]

You misread my intentions. I'm not arguing for or against any changes.

I'm just explaining the historical context of old changes, and I recognize that conditions have changed since then. The headmins can change the number to whatever they want. I'm just pointing out alternatives I feel should be considered.