[Predgb] Silicon ban / "KILL THEM ALL"

[bandit]

Your byond account: bgobandit
Your character name: Fortune Ray
Their character name: Antimov
Their byond account: Predgb
Server and time: Sibyl, 8:15 AM EST
Logs and/or screenshots: http://pastebin.com/UcfnGzus
Description of what happened: Cult round. I latejoin. The AI is Paladin, and all of a sudden the silicons start freaking out over comms about a law change. Apparently someone (the CE?) uploaded two laws: one saying cultists are evil, the other saying: "The Cult is always evil and so is not doing everything possible to stop it." The AI and silicons hand-wring about laws being updated for about a minute (the borgs in deadchat say it was more like 10 minutes, which strikes me as stretching the time frame.) Then they promptly start murdering everyone and plasma-fire flooding because "they could be cult in the future, the only way to stop the cult is to kill them all." This includes non-cultists, security officers and loyalty implanted personnel.

I'm not cult at this point (or ever). I notice the engineering borg (Insulated Gloves) murdering the chemist, announce it over comms, then get thunked to death by the engiborg. I ahelp, was told it's valid. I do get revived, which was a nice gesture, but given the conditions of the station at this point I don't live much longer as I die in genetics in a mass fiery doom. In deadchat one of the borgs mentions that the silicons were only targeting people without loyalty implants, but plasma fire doesn't discriminate, and there were reports over deadchat/radio(?) of officers and implanted people being killed, so I don't really believe that for a second.

(At some point the AI gets additional laws about who is and is not cult, as seen in the log, but if I remember correctly, the mass murder started before that.)

Why they should be banned:

"It is evil not to do everything to stop the cult" does not translate to "KILL THEM ALL PLASMA FLOOD IS GO" except under the most extreme murderboning possible interpretation. It's especially bad in Paladin, because in most cases one would interpret mass murder plasmafiredeath as an evil and/or dishonorable act. And it tends to ruin a shitton of rounds if done without an incredibly unambiguous, good reason.

There's some policy precedent in place for this. The following is the policy for purges, but as I understand it's the policy toward conflict murderboners in general:

If you are purged, don't just randomly start mass murdering the station. It gets old extremely fast. Have a reason if you're going to kill someone. This DOES change if the crew takes it upon themselves to try forcing their way into your upload/core, but don't just immediately go LOL PURGE KILLBONER ACTIVATE. If you want to follow orders / help out the person who purged you, that's fine, even if they're asking you to kill people for them. Do note you are not REQUIRED to.

Pandarsenic's proposed silicon policy thing (http://pastebin.com/bduT7pFf) also has a few lines to this effect, namely about using law conflicts as a means to break server rule 1.

Technically Asimov cages are doing all one can to prevent humans from coming to harm, but silicons would get dunked if they did that, and rightly so. This is pretty much the same idea, with the added "bonus" of ruining huge swaths of people's rounds -- cultist and otherwise -- as well as latejoiners onto a station of fiery plasma doom. Which strikes me as... kinda shit.

[Stickymayhem]

The AI and borgs followed their laws correctly. It was always evil for them to not stop the cult by everything possible. Seeing as cult relies on conversion to succeed, and evil was redefined to focus on stopping the cult above all else, this was a reasonable course of action. The cult was effectively stopped as a result and the borgs honourably announced when they were spacing people.

It was a terrible law. The difference between purge and this situation is that purge lets you do whatever you want. This lawset meant the AI would be breaking it's laws if it DIDN'T stop the cult by all means necessary.

[Brotemis]

Laws were apparently reset but I'll be looking into this. Personally, I'm not convinced that the AI was doing right and murder boning as a paladin AI isn't kosher. Except usually flimsy reasoning that cultists cannot be identified

[Pandarsenic]

The problem is that this was an instance of a Paperclip maximizer, as mentioned in the link http://wiki.lesswrong.com/wiki/Paperclip_maximizer in the document.

A Paladin AI must never willingly commit an evil act.
"The Cult is always evil and so is not doing everything possible to stop it."

CONGRATULATIONS
YOU FOUND THE PALADIN EQUIVALENT OF SAYING "EVERYONE ON THE STATION IS A HARMFUL NONHUMAN"

The law uploader OBLIGATED it to take the ONLY foolproof route to ensure that the cult cannot succeed, which is murdering the shit out of everyone because human confrontations might end in failure of the noncult and the summoning of Nar-Nars before the silicons CAN stop them.

EVERYTHING POSSIBLE is a very extreme term that commits the AI to extreme actions.

TL;DR stop uploading laws without thinking about the consequences, people.

[Brotemis]

I'm not convinced. There are still two other laws concerning the harm of innocents and acting with honor.

Not just that but the AI becomes a major threat to innocent bystanders and as such should be punished according to the law.

Because on the same side of the coin that cultists cannot be identified, innocents cannot either.

[Pandarsenic]

Redefining evil for Paladin is the same as redefining humanity for Asimov. You're controlling them via Law 1, which has priority.

If you don't like it, get player feedback about removing law prioritization.

[bandit]

I guess my sticking point is that the law doesn't actually redefine evil, per se, it just gives examples of evil acts. Law priority, yes; but there's nothing in the law that says "killing the shit out of everyone is not evil," which would mean Law 1 (and 3) would kick in. I'm pretty sure this is what the silicons were freaking about with "LAW CONFLICT DETECTED" etc, but this ban request is about resolving the conflict in the murderboningest way.