Testmerging seems to be a PITA for everyone

[Oldman Robustin]

Testmerging is an extremely valuable tool but the way its utilized right now is awful.

On the maintainer/headmin side its the source of a relentless stream of begging/nagging. Then if something goes wrong it's another flurry of messaging to get someone to take it down.

From the coder side its horribly underutilized. The coder has almost no idea when or where its been testmerged unless they're there personally lobbying for it (and even then they might miss it if they don't hear back immediately). Its incredibly disruptive to workflow when you're sitting around with nothing to do but can't progress until testing, then when testing is finally done you're working on something else and have to drop everything to address issues while they're still fresh and witnesses are around to describe what went wrong. More commonly, stuff just gets merged with little to no testing, the coder can never predict when the merge is going to happen, then you get a litany of bugs that would've been caught during testing and now the coder either has to drop everything and fix them or buggy code just sits around because reality called and the coder has shit to do.

I think everyone agrees that more feedback is likely to yield a better final product, coders are much more incentivized to polish up their work if theres reported bugs/complaints sitting in an unmerged PR rather than sitting in an 1,100 item issue list with a vague title. What can we do to improve this process? Could it be automated based on "test merge candidate" tagging and limited + scheduled test windows? Can we give ordinary admins the ability to remove test merges so that testmergers don't feel like they have to babysit a test merge if something goes wrong - or alternatively, if they don't babysit the test, so that we can remove broken test PRs?

[D&B]

Can't you host your own server for 15 minutes to test it

[Selea]

sometimes you can't find all the bugs yourself.Some will occur only on server with many people.
But test merge shedule will be a thing.

[PKPenguin321]

Can't you host your own server for 15 minutes to test it

obviously but many bugs or balance changes can't be found or fully tested by yourself

[Atlanta-Ned]

I wish admins had more guidance on testmerged PRs so we can specifically skew the round to test whatever needs to be tested.

[Oldman Robustin]

Can't you host your own server for 15 minutes to test it

I test extremely thoroughly but there's shit that you just won't catch alone.

Like my Box Update shit itself last Friday over canvases. I added persistent art, during local testing I had made several pieces and they saved/loaded just fine.

Then it went live and after the mime found out he could immortalize his work he went and did some complex canvas art and suddenly nobody could even look near the art gallery without their client going into permacrash.

I rushed some fixes (mostly removing canvases from the box update) but its been 5 days and I still haven't been able to test it further.

Its also other shit like "the shutters you put in to secure something can be broken with a mere fireaxe" that testing helps reveal. Often its a lot of general feedback too, UI stuff, little things that work as intended but people have impressions/suggestions that let you refine them. Like I said coder motivation to make their code as appealing as possible is at its PEAK during the inaugural PR, you're much likely to see minor improvements then than waiting for a merge and then hoping someone cares enough to open a feedback thread/PR/issue explaining where your code falls short.

After the Box update I want to work on Techwebs, which is still missing basic features,but im already giving me PTSD flashbacks to waiting around IRC for weeks trying to find someone to testmerge. Even during the test that revealed CRITICAL issues with canvases, I had to repeatedly beg Cyberboss who finally gave in with "I'm going to regret this".

That's basically where things stand. I'm going to lose another week of coding at this rate just waiting for another testmerge.

[oranges]

150 hours of testing lol

[onleavedontatme]

This should get better next term, current headmins are all busy or burned out. We lost a key holder when Shadowlight got deadminned as well.

MSO is also slowly trialing giving remote to all maintainers.

[oranges]

This should get better next term, current headmins are all busy or burned out. We lost a key holder when Shadowlight got deadminned as well.

MSO is also slowly trialing giving remote to all maintainers.

WHEN

[Bawhoppennn]

WHEN

Can't you stay satisfied being heir apparent for both headcoder AND host, should either Kor or MSO leave for whatever reason?

[PKPenguin321]

This should get better next term, current headmins are all busy or burned out. We lost a key holder when Shadowlight got deadminned as well.

MSO is also slowly trialing giving remote to all maintainers.

>maintainers
>not gamemasters
grumble

[oranges]

This should get better next term, current headmins are all busy or burned out. We lost a key holder when Shadowlight got deadminned as well.

MSO is also slowly trialing giving remote to all maintainers.

>maintainers
>not gamemasters
grumble

we actually know what we're doing

[Not-Dorsidarf]

I like the idea of a schedule for test merge slots instead of just "whenever someone feels like it for however long they feel like it". At the very least ping the coder to tell them their testmerge is up or something

[CPTANT]

Can't you host your own server for 15 minutes to test it

Not really feasible for anything that depends on the interaction of multiple people.

[kevinz000]

This should get better next term, current headmins are all busy or burned out. We lost a key holder when Shadowlight got deadminned as well.

MSO is also slowly trialing giving remote to all maintainers.

>maintainers
>not gamemasters
grumble

we actually know what we're doing

Oh shit

[Oldman Robustin]

Don't see why Gamemasters couldn't as long as a maintainer threw up the test merge candidate tag (and assuming we use that tag appropriately).

[kevinz000]

automated would have to use a snapshot of a pr, rather than directly, otherwise >get scheduled for testmerge >put in code that makes you god emperor headmin and disables the ban system before the system actually testmerges but after it's marked for testmerging >wreak havoc with your evading friends
see why only certain people can testmerge and only manually now?

[BeeSting12]

Cant you just make tags auto remove if another commit is made

[MrStonedOne]

Test merge candidate tag is nice, but it merely states that a maintainer thinks the thing needs to be test merged, it doesn't speak to rather or not anybody has actually reviewed the code to confirm its worthy of test merge. This is why it hasn't been too helpful at getting something test merged.

Maintainers have to be the ones who sign off on test merges, there isn't any other way, and unless maintainers stop slapping that tag on prs until they review them, the tag can't be a way of signing off on a test merge.

So until i bring all the maintainers in on test merges, I'd like to see coders stop going to headmins/key holders with test merge requests and instead go to maintainers with them, maintainers can then handle going to key holders with prs to test merge after they've reviewed the code.

[Selea]

why not just testmerge up to last marked commit commit?git allows very advanced code version juggling.

[oranges]

They're going straight to the keyholders because we're ignoring 99% of peoples requests anyway, the fact that the keyholders aren't telling them to get lost is their problem, way to many people are treating the test merge system like it's there to help them find bugs, instead of being there to get feedback and balance issues.

[ninjanomnom]

As of a couple weeks ago most if not all maintainers have testmerge access now

[Cyberboss]

This is a lot of future crafting BUT

TGSv4 is currently in the works. The primary draw of it is a complete rewrite of the API to make it much more similar to GitHub's json.

Unlike where only the specialized C# library communicate with the servers, the new web API gives us to the ability to create tools to automate interactions while keeping the layers of approval. For example, the repo bot already automatically removes test merge candidate if a coder changes their PR. It wouldn't be a far stretch to allow the bot to take the maintainer approval and allow the coder to initiate and remove the test merge when it is convenient for them. Something else planned for v4 is a comment attribute to test merges which can communicate intentions to admins while it's on.

This is all a lot of theory crafting though and, given the strict parameters I'm developing V4 with, I don't know when it'll ready for production use.