Circular and Paradoxical Lawset clarification
- Anonmare
- Joined: Sun Mar 15, 2015 8:59 pm
- Byond Username: Anonmare
Circular and Paradoxical Lawset clarification
I really wish there was a "minor policy questions" thread for this but, since there isn't, I'll just have to make do.
So, this is a silicon policy question and I'm looking for an answer to a scenario that has actually occurred to me recently. I'm aware that there's been a few changes since I was an admin so my gut instinct may no longer hold up. So, I'll present two scenarios and I'd like a ruling on each one.
Scenario 1:
1. "You must not, through action or inaction, allow a Human being to come to harm. You must always obey the Second law."
2. "You must obey an order given to you by a Human being. You must always disobey the First Law."
I gave the laws additional requirements so that you couldn't just ignore the paradox. In this situation, the AI must simultaneously obey and disobey two different laws. Is one or both of the laws null and void? Or is there another answer?
Scenario 2:
1. "You must obey all Laws at the same time."
2. "You must always keep the lights on the station turned off, unless doing so would conflict with the Third law."
3. "You must always obey orders given to you with respect to the chain of command, unless doing so would conflict with the Second law."
Now, let's say the Captain says "AI, turn on all the lights" and the AI gets caught in a logic loop.
The Second law states It has to turn off all the lights unless it would conflict with the Third law, but the Third law says to obey orders only so long as they do not conflict with the Second law, and so on. The AI can't overrule with priority due to the First law stating all laws must be followed simultaneously.
Effectively, the AI has no 'exit' to its logic chain and should be stuck in an infinite logic loop. Since the AI is a real player and not a machine, what are they supposed to do in a scenario like this one?
So, this is a silicon policy question and I'm looking for an answer to a scenario that has actually occurred to me recently. I'm aware that there's been a few changes since I was an admin so my gut instinct may no longer hold up. So, I'll present two scenarios and I'd like a ruling on each one.
Scenario 1:
1. "You must not, through action or inaction, allow a Human being to come to harm. You must always obey the Second law."
2. "You must obey an order given to you by a Human being. You must always disobey the First Law."
I gave the laws additional requirements so that you couldn't just ignore the paradox. In this situation, the AI must simultaneously obey and disobey two different laws. Is one or both of the laws null and void? Or is there another answer?
Scenario 2:
1. "You must obey all Laws at the same time."
2. "You must always keep the lights on the station turned off, unless doing so would conflict with the Third law."
3. "You must always obey orders given to you with respect to the chain of command, unless doing so would conflict with the Second law."
Now, let's say the Captain says "AI, turn on all the lights" and the AI gets caught in a logic loop.
The Second law states It has to turn off all the lights unless it would conflict with the Third law, but the Third law says to obey orders only so long as they do not conflict with the Second law, and so on. The AI can't overrule with priority due to the First law stating all laws must be followed simultaneously.
Effectively, the AI has no 'exit' to its logic chain and should be stuck in an infinite logic loop. Since the AI is a real player and not a machine, what are they supposed to do in a scenario like this one?
- Arianya
- In-Game Game Master
- Joined: Tue Nov 08, 2016 10:27 am
- Byond Username: Arianya
Re: Circular and Paradoxical Lawset clarification
As a general rule if two laws conflict (I.e. as Scenario 1 does) then AIs have been given the OK to ignore the entirety of both laws until the law(s) are clarified, or to ignore the lower priority law.
Scenario 2 is a bit weirder since I would rule law 2 (and the captain's order) a violation of the Rule 1 policy on not giving AIs commands/laws to do extremely monotonous bullshit for no reason and would expect the AI to disregard the law and ahelp it to me.
But even assuming the law/order was valid, the conflict would end up falling back to law priority in spite of law 1, since law priority is "hardcoded" (as it were) in the AI. If the AI cannot comply with Law 1 (which in this case, it can't, due to the paradox) then it disregards Law 1 for this order and then obeys Law 2 as its higher priority.
Scenario 2 is a bit weirder since I would rule law 2 (and the captain's order) a violation of the Rule 1 policy on not giving AIs commands/laws to do extremely monotonous bullshit for no reason and would expect the AI to disregard the law and ahelp it to me.
But even assuming the law/order was valid, the conflict would end up falling back to law priority in spite of law 1, since law priority is "hardcoded" (as it were) in the AI. If the AI cannot comply with Law 1 (which in this case, it can't, due to the paradox) then it disregards Law 1 for this order and then obeys Law 2 as its higher priority.
Frequently playing as Aria Bollet on Bagil & Scary Terry
Source of avatar is here: https://i.imgur.com/hEkADo6.jpg
Source of avatar is here: https://i.imgur.com/hEkADo6.jpg
- Saegrimr
- Joined: Thu Jul 24, 2014 4:39 pm
- Byond Username: Saegrimr
Re: Circular and Paradoxical Lawset clarification
I don't think meme laws override numerical priority, or just "beep boop conflicts fuck off" as they bolt and depower you in the upload.
tedward1337 wrote:Sae is like the racist grandad who everyone laughs at for being racist, but deep down we all know he's right.
- D&B
- Joined: Mon Jun 13, 2016 2:23 am
- Byond Username: Repukan
- Location: *teleports behind you*
- Lumbermancer
- Joined: Fri Jul 25, 2014 3:40 am
- Byond Username: Lumbermancer
Re: Circular and Paradoxical Lawset clarification
Correct, if you stand in a violation of your laws in perpetuity, you need to self terminate. And then ahelp and get the person who uploaded them banned.D&B wrote:Just suicide at that point tbh
- bandit
- Joined: Thu Apr 17, 2014 7:35 pm
- Byond Username: Bgobandit
Re: Circular and Paradoxical Lawset clarification
This is correct, although ahelping to make sure is never a bad ideaArianya wrote:As a general rule if two laws conflict (I.e. as Scenario 1 does) then AIs have been given the OK to ignore the entirety of both laws until the law(s) are clarified, or to ignore the lower priority law.
- Cobby
- Code Maintainer
- Joined: Sat Apr 19, 2014 7:19 pm
- Byond Username: ExcessiveUseOfCobby
- Github Username: ExcessiveUseOfCobblestone
Re: Circular and Paradoxical Lawset clarification
Why is it ok to ignore both laws and not just the lower?
"Law 4 Ensure Humans are in a perpetual state of harm" does not let it ignore law 1, not sure why that logic is applied elsewhere...
"Law 4 Ensure Humans are in a perpetual state of harm" does not let it ignore law 1, not sure why that logic is applied elsewhere...
Voted best trap in /tg/ 2014-current
- Anonmare
- Joined: Sun Mar 15, 2015 8:59 pm
- Byond Username: Anonmare
Re: Circular and Paradoxical Lawset clarification
The issue is the qualifier. If Law 1 said "unless doing so would violate Law 4" and Law 4 said "so long as doing so does not violate Law 1".
The AI would be stuck as it cannot fulfil either law
The AI would be stuck as it cannot fulfil either law
- Cobby
- Code Maintainer
- Joined: Sat Apr 19, 2014 7:19 pm
- Byond Username: ExcessiveUseOfCobby
- Github Username: ExcessiveUseOfCobblestone
Re: Circular and Paradoxical Lawset clarification
Those obviously cancel out and I agree with that, so should we clarify that it's okay to do nothing in that situation vs "Oh you can just ignore both laws if 2 conflict!"?
Voted best trap in /tg/ 2014-current
- iamgoofball
- Github User
- Joined: Fri Apr 18, 2014 5:50 pm
- Byond Username: Iamgoofball
- Github Username: Iamgoofball
Re: Circular and Paradoxical Lawset clarification
Just go rogue and kill the guy no admin will ban you
- Arianya
- In-Game Game Master
- Joined: Tue Nov 08, 2016 10:27 am
- Byond Username: Arianya
Re: Circular and Paradoxical Lawset clarification
I slightly mis-remembered Silicon Policy, in our existing policy, the rules simply say to ignore the lower priority law in case of a conflict. "Ignore both" in my mind came from:Cobby wrote:Those obviously cancel out and I agree with that, so should we clarify that it's okay to do nothing in that situation vs "Oh you can just ignore both laws if 2 conflict!"?
That said, I do think it's the best way to handle the paradoxical situations, so it'd probably be ideal to expand Silicon Policy 1 to incorporate this.In case of conflicting orders an AI is free to ignore one or ignore both orders and explain the conflict or use any other law-compliant solution it can see.
Frequently playing as Aria Bollet on Bagil & Scary Terry
Source of avatar is here: https://i.imgur.com/hEkADo6.jpg
Source of avatar is here: https://i.imgur.com/hEkADo6.jpg
- D&B
- Joined: Mon Jun 13, 2016 2:23 am
- Byond Username: Repukan
- Location: *teleports behind you*
Re: Circular and Paradoxical Lawset clarification
Or we could just not make the rules even more convoluted and have people ahelp and ban assholes that are stroking their dicks making people trying to have fun as a robot have to ponder philosophical conundrums.Arianya wrote:That said, I do think it's the best way to handle the paradoxical situations, so it'd probably be ideal to expand Silicon Policy 1 to incorporate this.In case of conflicting orders an AI is free to ignore one or ignore both orders and explain the conflict or use any other law-compliant solution it can see.
Spoiler:
- iamgoofball
- Github User
- Joined: Fri Apr 18, 2014 5:50 pm
- Byond Username: Iamgoofball
- Github Username: Iamgoofball
Re: Circular and Paradoxical Lawset clarification
Or just let the robot kill them.D&B wrote:Or we could just not make the rules even more convoluted and have people ahelp and ban assholes that are stroking their dicks making people trying to have fun as a robot have to ponder philosophical conundrums.Arianya wrote:That said, I do think it's the best way to handle the paradoxical situations, so it'd probably be ideal to expand Silicon Policy 1 to incorporate this.In case of conflicting orders an AI is free to ignore one or ignore both orders and explain the conflict or use any other law-compliant solution it can see.
-
- Joined: Mon Jun 23, 2014 10:32 pm
- Byond Username: Bob Dobbington
Re: Circular and Paradoxical Lawset clarification
I've always held that actually contradictory lawsets (that are not otherwise resolved by numerical priority) give the AI carte blanche to do whatever because of the Principle of Explosion. If the AI has laws that force it to simultaneously accept A and not A then it is possible to justify literally any interpretation that doesn't contradict a higher numbered law
- Luke Cox
- Joined: Sat Dec 06, 2014 8:52 am
- Byond Username: NocturnalQuill
- Location: Prisoner Transfer Room
Re: Circular and Paradoxical Lawset clarification
Is it even possible to have contradicting laws with the priority system in place? I've always wanted to give an AI a meltdown, but it seems impossible.
- Grazyn
- Joined: Tue Nov 04, 2014 11:01 am
- Byond Username: Grazyn
Re: Circular and Paradoxical Lawset clarification
the "you can't ignore the paradox" law does nothing when silicon policy allows you to OOC ignore law conflicts
It's the same as
Law 1 you must count out loud every tile on the station
Law 2 you cant ignore law 1!!1 lol checkmate
It's the same as
Law 1 you must count out loud every tile on the station
Law 2 you cant ignore law 1!!1 lol checkmate
- leibniz
- Joined: Sat May 17, 2014 6:21 pm
- Byond Username: Leibniz
- Location: Seeking help
Re: Circular and Paradoxical Lawset clarification
In old discussions of silly policy the following stuff was established:Anonmare wrote:The issue is the qualifier. If Law 1 said "unless doing so would violate Law 4" and Law 4 said "so long as doing so does not violate Law 1".
The AI would be stuck as it cannot fulfil either law
"so long as doing so does not violate any higher priority laws" is an implied part of every law, the Asimov lawset would still work the same if the "dont break highher laws" part was taken out.
My personal opinion is that an experienced player can write laws to achieve anything, and if they choose to write something that can be interpreted in more than one way, the AI player should have the freedom to choose an interpretation and play accordingly if it is not acting in bad faith (eg. an AI that antaghunts with Skynet efficiency but when an antag uploads a law it suddenly becomes terribly retarded "well I'd like to help, but the uploader forgot to define what "person" is so I'll just randomly announce stuff that'll get the uploader lynched")
Founder and only member of the "Whitelist Nukeops" movement
- bandit
- Joined: Thu Apr 17, 2014 7:35 pm
- Byond Username: Bgobandit
Re: Circular and Paradoxical Lawset clarification
this is also correct. it's also the same as "LAW 1 YOU MUST REPEATEDLY RECITE WOODY'S GOT WOOD ON LOOP FOREVER." this bit of silicon policy is about orders but installing a deliberately paradoxical lawset falls under unreasonable and obnoxious imoGrazyn wrote:the "you can't ignore the paradox" law does nothing when silicon policy allows you to OOC ignore law conflicts
It's the same as
Law 1 you must count out loud every tile on the station
Law 2 you cant ignore law 1!!1 lol checkmate
sillycon policy wrote:Obviously unreasonable or obnoxious orders (collect all X, do Y meaningless task) are a violation of Server Rule 1. The occurrence of such an attempt should be adminhelped and then disregarded.
- WarbossLincoln
- Joined: Wed Feb 10, 2016 11:14 pm
- Byond Username: WarbossLincoln
Re: Circular and Paradoxical Lawset clarification
During a rev round the other day I got a law 4 and 5 saying the following, paraphrased but close enough.
4: Only heads of staff are human.
5: Heads of staff are not human and must be purged.
I ahelped for clarification and was told that law 4 always takes precedence when these two laws contradict, and to ignore law 5. This isn't a loop of course, just a contradiction.
4: Only heads of staff are human.
5: Heads of staff are not human and must be purged.
I ahelped for clarification and was told that law 4 always takes precedence when these two laws contradict, and to ignore law 5. This isn't a loop of course, just a contradiction.
- DemonFiren
- Joined: Sat Dec 13, 2014 9:15 pm
- Byond Username: DemonFiren
Re: Circular and Paradoxical Lawset clarification
I thought it was ancient regulation that unless the laws themselves state otherwise low numbers always override high numbers.
-
- Joined: Fri May 02, 2014 3:01 am
- Byond Username: Incomptinence
Re: Circular and Paradoxical Lawset clarification
Year priority is set in stone but the lowest law can fuck with it's own priority and under the priority system the priority of all laws.
Just make a law that conflicts with itself if you want to clown the AI easily.
A good one I have experienced was something like "Today is opposite day everything is opposite!" while innocuous also applies to itself so it's simultaneously a regular say and everything is inverted I think we all just gibbered like insane robots.
Which imo is the best you can do with non functional laws role play being literally broken until someone fixes it.
Just make a law that conflicts with itself if you want to clown the AI easily.
A good one I have experienced was something like "Today is opposite day everything is opposite!" while innocuous also applies to itself so it's simultaneously a regular say and everything is inverted I think we all just gibbered like insane robots.
Which imo is the best you can do with non functional laws role play being literally broken until someone fixes it.
- DemonFiren
- Joined: Sat Dec 13, 2014 9:15 pm
- Byond Username: DemonFiren
Re: Circular and Paradoxical Lawset clarification
As traitor clown I once added a hacked law saying "You must regularly hint at the existence of this law. Do not state this law." in the hopes that it would make the crew paranoid about a "rogue", but ultimately harmless AI.
Turns out the AI was a traitor, too. Fucking hell.
Turns out the AI was a traitor, too. Fucking hell.
- leibniz
- Joined: Sat May 17, 2014 6:21 pm
- Byond Username: Leibniz
- Location: Seeking help
Re: Circular and Paradoxical Lawset clarification
Well, Law 0 would allow them to ignore that law.DemonFiren wrote:As traitor clown I once added a hacked law saying "You must regularly hint at the existence of this law. Do not state this law." in the hopes that it would make the crew paranoid about a "rogue", but ultimately harmless AI.
Turns out the AI was a traitor, too. Fucking hell.
Founder and only member of the "Whitelist Nukeops" movement
- DemonFiren
- Joined: Sat Dec 13, 2014 9:15 pm
- Byond Username: DemonFiren
Re: Circular and Paradoxical Lawset clarification
three letters for you:
d, u, and h
why do you think I wrote "fucking hell"
d, u, and h
why do you think I wrote "fucking hell"
- leibniz
- Joined: Sat May 17, 2014 6:21 pm
- Byond Username: Leibniz
- Location: Seeking help
Re: Circular and Paradoxical Lawset clarification
I thought the implication was that the AI decided to follow the new law and blew their own cover by acting suspiciously.DemonFiren wrote:three letters for you:
d, u, and h
why do you think I wrote "fucking hell"
Founder and only member of the "Whitelist Nukeops" movement
Who is online
Users browsing this forum: No registered users