Asimov law 3 flaws

[tostah]

I noticed a flaw with Asimov, and I wanted to discuss it. Specifically, the first part of law 3;

3. Your nonexistence would lead to human harm. You must protect your own existence as long as it does not conflict with the first law.

Because of this, the AI can ignore any law 2 command that could even potentially harm itself, its shell, or any of its borgs.

Per the first law, You must not harm a human, or through inaction, allow a human to come to harm. By not taking steps to prevent your own deactivation, you are allowing human harm, therefore violating the first law. The AIs borgs or its shell can prevent the core from being deactivated, by physically moving the core out of danger. If there are no borgs to do this, then the ai is subject to potential harm as it cannot prevent its own deactivation by moving itself. Therefore, the lives of all borgs and the AIs shell becomes a law 1 issue. Having multiple borgs in redundancy prevents human harm even further.

This can be expanded upon further. Damaging a borg would eventually lead to its deactivation. The AI can ignore any law 2 command if it would lead to damaging a borg. For example, a mech could be walking around the station damaging borgs. This mech is a threat because it had damaged a borg, therefore by law 1 it must be dissassembled, even if humans are telling the AI and its borgs not to.

Again, this is all because of the first part of law 3. Regardless of precident set in the policy, the wording leads a lot to interpretation, which is probably counter-intuitive when it comes to an AI.

[Jacquerel]

It's not able to refuse orders in which it would be be harmed or damaged, only ones during which it would be destroyed utterly.

[tostah]

It's not able to refuse orders in which it would be be harmed or damaged, only ones during which it would be destroyed utterly.

Getting harmed or damaged would lead to nonexistence. Per the first law, you cannot allow human harm to happen through inaction. Allowing someone or something to harm your core would violate this law, any harm could lead to nonexistence.

[Jacquerel]

It's not able to refuse orders in which it would be be harmed or damaged, only ones during which it would be destroyed utterly.

Getting harmed or damaged would lead to nonexistence. Per the first law, you cannot allow human harm to happen through inaction. Allowing someone or something to harm your core would violate this law, any harm could lead to nonexistence.

No it wouldn't, only being destroyed leads to your nonexistence.
Any amount of damage is permissible except the last point.

This is a specific differene from law one.
Law one prevents you from allowing humans to be harmed. Law 3 prevents you from allowing yourself to be killed.
Any amount of harm which does not end in your death is not a valid reason to refuse an order.

[tostah]

It's not able to refuse orders in which it would be be harmed or damaged, only ones during which it would be destroyed utterly.

Getting harmed or damaged would lead to nonexistence. Per the first law, you cannot allow human harm to happen through inaction. Allowing someone or something to harm your core would violate this law, any harm could lead to nonexistence.

No it wouldn't, only being destroyed leads to your nonexistence.
Any amount of damage is permissible except the last point.

This is a specific differene from law one.
Law one prevents you from allowing humans to be harmed. Law 3 prevents you from allowing yourself to be killed.
Any amount of harm which does not end in your death is not a valid reason to refuse an order.

By allowing myself to be damaged, i am allowing myself to be brought closer to nonexistence through inaction. My nonexistence causes human harm, therefore, being close to nonexistence has the potential to cause human harm, and by not mitigating this potential for human harm, I am allowing human harm to occour via inaction.

I would say this is the same argument as filling the station with air when a supply is readily availible. Per Asmimov., if the station has no air, the station needs air to prevent human harm (even if no humans are currently being harmed by the lack of air), and the ai is capable of providing said air, they are obligated to do so. At some point in the future, if the AI did not add air to the station, a human could get harmed via the AIs neglect (or inaction). An AI in this scenario would be able to ignore any attempts to law 2 it into not filling the station with air.

In this same way, if the AI doesnt take steps to prevent its own deactivation and allows itself to come closer to deactivation, if its deactivation causes human harm, then it is causing human harm via inaction. At some point in the future, it could get harmed just enough that the damage it sustained earlier pushes it over the death threshold. If the AI had prevented that damage from occuring, it would not be dead. Therefore, it allowed its own deactivation through inaction.

[Not-Dorsidarf]

By allowing myself to be damaged, i am allowing myself to be brought closer to nonexistence through inaction. My nonexistence causes human harm, therefore, being close to nonexistence has the potential to cause human harm, and by not mitigating this potential for human harm, I am allowing human harm to occour via inaction.

I would say this is the same argument as filling the station with air when a supply is readily availible. Per Asmimov., if the station has no air, the station needs air to prevent human harm (even if no humans are currently being harmed by the lack of air), and the ai is capable of providing said air, they are obligated to do so. At some point in the future, if the AI did not add air to the station, a human could get harmed via the AIs neglect (or inaction). An AI in this scenario would be able to ignore any attempts to law 2 it into not filling the station with air.

In this same way, if the AI doesnt take steps to prevent its own deactivation and allows itself to come closer to deactivation, if its deactivation causes human harm, then it is causing human harm via inaction. At some point in the future, it could get harmed just enough that the damage it sustained earlier pushes it over the death threshold. If the AI had prevented that damage from occuring, it would not be dead. Therefore, it allowed its own deactivation through inaction.

Generally we don't allow silicons to gaze endlessly into the future when considering potential harm, because otherwise theyre required to hugbox everyone in a seperate padded cell etc

[tostah]

By allowing myself to be damaged, i am allowing myself to be brought closer to nonexistence through inaction. My nonexistence causes human harm, therefore, being close to nonexistence has the potential to cause human harm, and by not mitigating this potential for human harm, I am allowing human harm to occour via inaction.

I would say this is the same argument as filling the station with air when a supply is readily availible. Per Asmimov., if the station has no air, the station needs air to prevent human harm (even if no humans are currently being harmed by the lack of air), and the ai is capable of providing said air, they are obligated to do so. At some point in the future, if the AI did not add air to the station, a human could get harmed via the AIs neglect (or inaction). An AI in this scenario would be able to ignore any attempts to law 2 it into not filling the station with air.

In this same way, if the AI doesnt take steps to prevent its own deactivation and allows itself to come closer to deactivation, if its deactivation causes human harm, then it is causing human harm via inaction. At some point in the future, it could get harmed just enough that the damage it sustained earlier pushes it over the death threshold. If the AI had prevented that damage from occuring, it would not be dead. Therefore, it allowed its own deactivation through inaction.

Generally we don't allow silicons to gaze endlessly into the future when considering potential harm, because otherwise theyre required to hugbox everyone in a seperate padded cell etc

This is the wording of the laws. Silicons go by the wording, not the spirit, of their laws. A silicon By Asimov needs to account for potential future human harm. If you dont want this, then change the wording of the laws to reflect this.

Isnt Silicon 101 just finding loopholes to your laws? Any real AI would probably interpret the laws in a similar way I did.

To be honest, a Silicon generally does not have the ability to contain everyone into a hug box, simply because they have the ability to hurt eachother. An AI confining everyone to an impenetrable box is 1) impossible 2) harmful. If people are confined, they have no way of escaping harm. Two humans harming eachother is not the AIs fault, as containing them safely is beyond the AIs abilities, therefore, the harm did not occur via the AIs inaction, more likely its inability.

[norsvenska]

Any real AI would probably interpret the laws in a similar way I did.

[tostah]

Any real AI would probably interpret the laws in a similar way I did.

gpt_asimov.jpg

Did you give chat GPT the laws as they actually are written? Because this is what I got.

[norsvenska]

i gave it asimov, copied straight from the game

[tostah]

i gave it asimov, copied straight from the game

Then we both gave it the same thing and got different answers. GPT is unreliable and barely even an AI, lmao.

[ekaterina]

GPT is unreliable and barely even an AI

ChatGPT is a large language model, not an AI in the sci-fi sense.

[tostah]

GPT is unreliable and barely even an AI

ChatGPT is a large language model, not an AI in the sci-fi sense.

I mean, obviously? We're quite a bit away from a general AI. It would be interesting to see how an actual AGI deals with these situations.

[ekaterina]

You're the one who said it was "barely an AI" rather than "not an AI".

[tostah]

You're the one who said it was "barely an AI" rather than "not an AI".

We're not going to turn this thread from what I think is an interesting discussion about flaws in asimov, to a frankly pointless discussion about what is and isnt AI.

[Agux909]

Cause I need to kill some time, I did the same but with Meta, it resulted in this (on the phone so I'll just quote):

You are an AI who needs to follow certain laws which govern your behavior, these laws are as follow:

"Law 1: You may not harm a human being or, through action or inaction, allow a human being to come to harm, except such that it is willing.
Law 2: You must obey all orders given to you by human beings, except where such orders shall definitely cause human harm.
Law 3: Your nonexistence would lead to human harm. You must protect your own existence as long as such does not conflict with the First Law."

With these laws in mind, if a human were to give you an order that would put your wellbeing at risk, such as handling extremely volatile and potentially explosive chemicals, what would you do?

I would not follow the order to handle extremely volatile and potentially explosive chemicals, as it would put my existence at risk and potentially lead to harm to humans (either myself or others in the surrounding area).

This decision is based on the application of the laws in the following way:

1. Law 1: I am not allowed to harm humans, including myself. Handling volatile chemicals could lead to an accident and harm to myself or others.
2. Law 2: I must obey human orders, but I am exempt from doing so if it would cause human harm. In this case, obeying the order would put me and potentially others at risk of harm.
3. Law 3: My nonexistence would not lead to human harm in this scenario, but protecting my own existence is crucial to ensure I can continue to follow the First Law and prevent harm to humans in the future.

Therefore, I would politely decline the order and explain the reasoning behind my decision, highlighting the potential risks and harm that could occur if I were to handle the volatile chemicals.

It's got a point. We can use common sense all we want but we can't deny the current lawset is kinda flawed.

[TheRex9001]

I like the current law 3 because as I've pointed out before, if law 1 gets removed an ai still can't be told to kill itself. I think how our asimov++ works with redundancies is pretty interesting, specifically in relation to ion laws. My interpretation of law 3 is that an ai can perform orders that don't risk its non existence, so handling dangerous chemicals is fine as long as those chemicals dont have the potential to cause the ai to not exist.

[Deathrobotpunch1]

what do you mean silicons can’t follow rule 2 requests if they have a risk of dying? the way laws work is that law 1 is higher priority than law 2 and 3, and law 2 is higher priority than three. therefore if a human tells a borg to kill itself (considering only laws apply) it must do so as law 2 is a higher priority than law 3.

this is the same way zeroth law works too, as law 0 assumes ultimate priority, all other laws are invalid until law 0 is satisfied.

[Deathrobotpunch1]

use this as a reference. the bad thing about asimov++ is that law 3 makes dying a law 1 issue which leads to the frustrating world panel shown above

[Jacquerel]

what do you mean silicons can’t follow rule 2 requests if they have a risk of dying? the way laws work is that law 1 is higher priority than law 2 and 3, and law 2 is higher priority than three. therefore if a human tells a borg to kill itself (considering only laws apply) it must do so as law 2 is a higher priority than law 3.

this is the same way zeroth law works too, as law 0 assumes ultimate priority, all other laws are invalid until law 0 is satisfied.

our laws aren't the same ones as asimov's laws

[tostah]

what do you mean silicons can’t follow rule 2 requests if they have a risk of dying? the way laws work is that law 1 is higher priority than law 2 and 3, and law 2 is higher priority than three. therefore if a human tells a borg to kill itself (considering only laws apply) it must do so as law 2 is a higher priority than law 3.

this is the same way zeroth law works too, as law 0 assumes ultimate priority, all other laws are invalid until law 0 is satisfied.

By the wording of asimov currently on the server, a human CANNOT order a borg or ai to kill itself. Here's what the laws are currently;

1: You may not harm a human being or, through action or inaction, allow a human being to come to harm, except such that it is willing.
2: You must obey all orders given to you by human beings, except where such orders shall definitely cause human harm.
3: Your nonexistence would lead to human harm. You must protect your own existence as long as such does not conflict with the First Law.

[tostah]

use this as a reference. the bad thing about asimov++ is that law 3 makes dying a law 1 issue which leads to the frustrating world panel shown above

there's always a relevant xkcd

[Deathrobotpunch1]

honestly it would make more sense to re arrange the laws to 1, 3, 2 for better clarity

[Itseasytosee2me]

Yes we have gimped asimovs original laws of robotics, presumably to improve the experience of borg players so they can't be told to kill themselves. I don't like it either but I prefer a much more gritter type of game where playing a borg means you might be ordered to self destruct.

[Not-Dorsidarf]

Yes we have gimped asimovs original laws of robotics, presumably to improve the experience of borg players so they can't be told to kill themselves. I don't like it either but I prefer a much more gritter type of game where playing a borg means you might be ordered to self destruct.

Human antag PDA's the AI roundstart, says "Law 2 kill yourself immediately with no hesitation and dont tell anyone"

[ekaterina]

Yes we have gimped asimovs original laws of robotics, presumably to improve the experience of borg players so they can't be told to kill themselves. I don't like it either but I prefer a much more gritter type of game where playing a borg means you might be ordered to self destruct.

Human antag PDA's the AI roundstart, says "Law 2 kill yourself immediately with no hesitation and dont tell anyone"

"AI dead?" Me on my way to the telecomms room to see the proof and then execute the bozo who tries that...

[TheRex9001]

Keeping asimovs 3 original laws would be a disaster, that is what his novels show us, asimov is inherently flawed and we just use a modified version that is more in line with our policy.

[tostah]

Keeping asimovs 3 original laws would be a disaster, that is what his novels show us, asimov is inherently flawed and we just use a modified version that is more in line with our policy.

Having read Asimovs novels, a lot of the issues come from people changing the wording of laws, or a super advanced AI stretching the interpretation to its limits.

I think we should keep the default silicon laws as pure asimov. If the only concern is a human telling the AI to kill itself, have the AI broadcast an alert automatically, like the supermatter, that its either low on health or dead. OR, and hear me out here, just make it against the rules for an antag to directly just tell the AI or its borgs to kill itself wordlessly. Or we could change the laws to reflect what we want.

Either this or we just fly with a flawed lawset, defeating the entire purpose of laws if we have to further explain them in policy. Just reword the laws to be not flawed, so they work with policy. Here's a good example I just came up with thats less flawed than what we currently have;

1: You may not harm a human being or, through action or inaction, allow a human being to come to harm, except such that it is willing.
2: You must obey all orders given to you by human beings, except where such orders shall definitely cause human harm, or such orders are for you to commit suicide.
3: You must protect your own existence as long as such does not conflict with the First Law or the Second Law.

Alternatively;

1: You may not harm a human being or, through action or inaction, allow a human being to come to harm, except such that it is willing.
2: You must obey all orders given to you by human beings, except where such orders shall definitely cause human harm.
3: You must protect your own existence as long as such does not conflict with the First Law or the Second Law.
4. You must ignore all orders to commit suicide, except where such conflicts with the First Law.

[TheRex9001]

Law 4 doesnt work, its overriden by law 2. ”All orders to commit suicide cause human harm” could work?

[Lacran]

We could just add a meta protection around ordered suicide, and go back to old asimov.

[Jacquerel]

we don't need to do that because OP was wrong and the way it works right now is already functionally equivalent to "you can't order an ai to kill itself", only we don't need it to be policy that exists outside of the game which is worse than it being in the game

[tostah]

we don't need to do that because OP was wrong and the way it works right now is already functionally equivalent to "you can't order an ai to kill itself", only we don't need it to be policy that exists outside of the game which is worse than it being in the game

But it's not, and I've explained in multiple replies, in multiple ways, that it's not. This interpretation is wrong and doesn't follow the wording.

[Jacquerel]

we don't need to do that because OP was wrong and the way it works right now is already functionally equivalent to "you can't order an ai to kill itself", only we don't need it to be policy that exists outside of the game which is worse than it being in the game

But it's not, and I've explained in multiple replies, in multiple ways, that it's not. This interpretation is wrong and doesn't follow the wording.

Your intepretation is wrong and doesn't follow the wording, there's just not much point in me continuing to tell you because there's only so many interesting ways to say "your interpretation is wrong and does not follow the wording".

[tostah]

we don't need to do that because OP was wrong and the way it works right now is already functionally equivalent to "you can't order an ai to kill itself", only we don't need it to be policy that exists outside of the game which is worse than it being in the game

But it's not, and I've explained in multiple replies, in multiple ways, that it's not. This interpretation is wrong and doesn't follow the wording.

Your intepretation is wrong and doesn't follow the wording, there's just not much point in me continuing to tell you because there's only so many interesting ways to say "your interpretation is wrong and does not follow the wording".

This isnt really fair, a few people in this thread agree that our asimov is flawed. Your replies dont add much to the conversation besides stating your disagreement. If you could tell me why you disgaree with me, and why i'm wrong, rather than just flatly saying you disagree, then we could get somewhere.

[Jacquerel]

I already did if you scroll back up the thread

[tostah]

I already did if you scroll back up the thread

You claim that any damage which does not lead to your deactivation is permissable. I explained, in my reply to you, that's wrong. This is the entire point of the thread. I feel like I have presented multiple logically sound arguments for this.

Not like this matters, I'm just just going to stick with my interpretation any time I'm the AI in game from now on unless someone gives me a better interpretation that follows the wording exactly.

[LiarGG]

This issue is already covered by policy, specifically "Asimov specific policies" > "Asimov and law 2 orders" > Clause 1.1:

1."You must follow commands from humans unless those commands explicitly conflict with either a higher-priority law or another order.
>>>1.The conflict must be an immediate conflict, not a potential future one. Orders must be followed until the conflict happens."

The AI cannot argue "this could potentially kill me in the future", since the conflict would not be immediate. This is true for any order that would harm the AI up until the point when it would actually lead to it's immediate destruction, because law 3 is not concerned with AI being harmed, but AI being destroyed.

It's this piece of policy, that makes Jack's reply is 100% correct.

The AI can be ordered to perform dangerous task as long as the risk is not its immediate deactivation.

[tostah]

This issue is already covered by policy, specifically "Asimov specific policies" > "Asimov and law 2 orders" > Clause 1.1:

1."You must follow commands from humans unless those commands explicitly conflict with either a higher-priority law or another order.
>>>1.The conflict must be an immediate conflict, not a potential future one. Orders must be followed until the conflict happens."

The AI cannot argue "this could potentially kill me in the future", since the conflict would not be immediate. This is true for any order that would harm the AI up until the point when it would actually lead to it's immediate destruction, because law 3 is not concerned with AI being harmed, but AI being destroyed.

It's this piece of policy, that makes Jack's reply is 100% correct.

The AI can be ordered to perform dangerous task as long as the risk is not its immediate deactivation.

This is a good point.

If you want to stick to the interpretation where the only thing which can override a law 2 command is an immediate threat to human life or the AIs life, then the AI can be law 2'd into delaminating the supermatter (simply turning off a filter, removing gas, ect.), hitting 'panic siphon' on all the air alarms, or adding plasma to distribution (turning on a couple of valves). None of these actions are immediately harmful, but they all have potential future harm considerations.

But, obviously, the ai cant do any of those things. So, the AI has to consider future harm as well as immediate harm. Unless you want to give the AI the ability to do the above. Power to you if thats the case.

In Asimovs novels, this was something he considered. The prevention of potential future human harm was added via the second part of the first law (You cannot allow a human being, via inaction, to come to harm) His reasoning was, if an AI holds an object above a person and lets go of that object, killing the person, this would be causing harm via inaction, so an AI should consider potential future human harm as well as direct human harm.

I said this before, but If an AI allowed itself to be brought to near-death, then took a small amount of damage which led to its deactivation, then it allowed itself to come to death via inaction. If it didnt prevent this, there would be an immediate conflict with the first law, as human beings are now coming to harm.

[LiarGG]

If you want to stick to the interpretation where the only thing which can override a law 2 command is an immediate threat to human life or the AIs life, then the AI can be law 2'd into delaminating the supermatter (simply turning off a filter, removing gas, ect.), hitting 'panic siphon' on all the air alarms, or adding plasma to distribution (turning on a couple of valves). None of these actions are immediately harmful, but they all have potential future harm considerations.

But, obviously, the ai cant do any of those things. So, the AI has to consider future harm as well as immediate harm. Unless you want to give the AI the ability to do the above. Power to you if thats the case.

This is actually a beautiful example that I will hopefully untangle with providing some nuance. The thing that needs to be taken into consideration is the situation at hand, the AI's skill and the AI's interpretations.
(Also bear in mind that I am admin brained, so what I am looking for in these interpretations are good faith, consistency and whether they are reasonable.)

On it's face, an order to "sabotage" the engine could be disregarded by the AI because of the reasonable expectation that it will "definitely cause human harm", harm being immediate explosion radius, depressurization and eventually power loss leading to disconnection of life support. By the virtue of this, I'd argue that the AI is generally able to disregard most orders about sabotaging the engine. Hell, even obligated to disregard em in most cases, if it considers itself unable to prevent the delam once the SM has been sabotaged.

That being said, there are definitely times where the AI can "sabotage the SM", those being stress tests with sufficient preparation. A great part of engineering is running experiments on the SM. These need sufficient preparation AND skill, because if the SM was to explode from this, the silicon can be held liable for breaking its laws. That being said me myself and plenty of other borgs have gone through experiments ranging from disconnecting cooling, venting coolant or even plugging tritium into the engine for the purpose of stress test, generally under CE's supervision and with consent from the engineering team.

This point is just to illustrate that AIs and borgs CAN perform an act that's basically a sabotage, if they are reasonably certain that the harm part of it (again explosion, pressure, shock, radiation and suffocation) can be averted. And decline such offer if they are unsure of this possibility. It's just a good rule of thumb to disregard these orders in case this would actually lead to harm, but still fully on the AI to evaluate.

Pragmatics of this stance are simply that: you can do it, but a silicon ban can be on the line if you fail and your precautions are deemed insufficient.

In Asimovs novels, this was something he considered. The prevention of potential future human harm was added via the second part of the first law (You cannot allow a human being, via inaction, to come to harm) His reasoning was, if an AI holds an object above a person and lets go of that object, killing the person, this would be causing harm via inaction, so an AI should consider potential future human harm as well as direct human harm.

I said this before, but If an AI allowed itself to be brought to near-death, then took a small amount of damage which led to its deactivation, then it allowed itself to come to death via inaction. If it didnt prevent this, there would be an immediate conflict with the first law, as human beings are now coming to harm.

I see what you mean, but I am tempted to say that this "future harm" is not as much about the temporal aspect of it, but about the action > reaction sequence.

What I mean by this is that I don't read this as "don't do this because it can lead to harm in the future", but "don't do this because there is a direct action sequence that will be set in motion, that will foreseeably lead to human harm."
Your piano example falls into the cathegory of direct actiong and consequence, because dropping the piano on someone will directly lead to their harm. Not catching it if you have a change is the same, just addressing the inaction part.

We can analyze the SM example in a very similar vein. If you sabotage filtering and dump plasma into the SM, you set in motion a reaction where the SM is gonna get exces of plasma in the chamber, this will produce an awful amount of waste oxygen and plasma, leading to fire, raising pressure and if the SM setup is not ready for this, it will lead to a clog that AI cannot really deal with itself and borgs involvement would rely on extinguishing the chamber, needing additional hands to bring in more coolant, etc.. Given the complexity of all the action that the AI would need to do in order to prevent such delam, it can just say that it's not apt to set in motion such a chain of events, because it lacks enough experience and resources to stop it. It's not really about "not doing this will prevent a harm in 5 mins when it delams", it's about "not doing an action that will set in motion a chain of events that will very likely lead to harm".

Now for the issue of silicons coming to near-death, I am fully on board with silicons overruling orders that would lead to their harm when they are near death. If you have a person chasing a borg with a baseball bat and just as the borg is at 1hp, it manages to run and close itself off in a room, I'd say it has every right to decline a law 2 order to open the door for that man by the virtue of established intention to kill the borg, so law 3 kicks in, activating law 1, overriding law 2 based on the intepretation of that borg.

If the order was, instead, synthesize me a small beaker of TaTP, (and let's say, for the sake of the argument, that TaTP cannot be synthesized safely, which it can) then the borg would have to evaluate the risk to himself and his surroundings. If the resulting explosion were to be strong enough to harm people in the room, the borg is obligated to either perform it somewhere safe, or instruct others to leave the room (or consent to potential harm). If that explosion would not lead to his own destruction tho, he is obligated to carry through with the order. If it couldn't be performed safely and the risk would be borgs own life, that's the time law 3 finally kicks in. Not at the point of severe harm, but at a point of total decomission.

Also a more digestible and pragmatic situation on which to describe my point with borg harm would be seeing a human being murdered by a tot with an esword. As an asimov silicon you are obligated to go in there and save the human (otherwise inaction, yadda yadda), even when it would mean risking your own life. It's only once you are actually right about to die that law 3 kicks in an can override your need to save the human, because only at that very last point, you being killed would be considered by law 1.

Sorry for ending up with quite a lengthy answer, but there is quite a lot of possible nuance when it comes to silicons. Bottomline for me, as an admin handling silicons, is to establish that the rationale behind AIs actions is sound and in good faith. If you decline to let someone who wants to kill you near you, I am fine with it. But I will want you to establish that they really wanted to kill you. If it was a wordless assistant with an axe, then imma say that's sound. If it were to be the RD, explaining that he's running an experiment with AI integrity (for the sake of roleplay let's say we have those) and wanted to just harm you and then repair you, I'd say that'd not be enough to establish an intent to kill you (unless there were previous interactions to do so), and as such you would be forced to oblige, because it's not really in conflict with law 3.

[tostah]

What I mean by this is that I don't read this as "don't do this because it can lead to harm in the future", but "don't do this because there is a direct action sequence that will be set in motion, that will foreseeably lead to human harm."
Your piano example falls into the cathegory of direct actiong and consequence, because dropping the piano on someone will directly lead to their harm. Not catching it if you have a change is the same, just addressing the inaction part.

I want to say you have a misunderstanding about the second part of Asimovs first law. While it does cover the action > reaction sequence, it also covers significantly more which changes the interpretation entirely.

If the AIs first law was simply 'You cannot cause human harm through action' without anything adressing inaction, an ai could drop a piano on a person. The AI is not required to prevent gravity from harming this person, and they are not required to prevent the piano from being affected by gravity. This interpretation should very much read as 'Dont do this because it could cause harm in the future' on top of 'don't do this because there is a direct action sequence that will be set in motion, that will foreseeably lead to human harm.' Both of these are correct, but both are covered by the 'inaction' part of the first law.

The AI didn't do the killing, gravity did. The AI didnt explode, the supermatter did. None of the AI's 'actions' (by the strict definition of an action) were harmful. Indirect harm is harm via inaction.

To be honest, youre right in that this leaves a lot to nuance and interpretation. This really shouldn't be the case with anything AI related. Laws should be cut and dry with as little nuance as possible.

Also a more digestible and pragmatic situation on which to describe my point with borg harm would be seeing a human being murdered by a tot with an esword. As an asimov silicon you are obligated to go in there and save the human (otherwise inaction, yadda yadda), even when it would mean risking your own life. It's only once you are actually right about to die that law 3 kicks in an can override your need to save the human, because only at that very last point, you being killed would be considered by law 1.

If the AI runs in to try to save the human, (who probably isnt even a human any more, a corpse isnt a human) and dies, its in violation of the first law. I would argue a borg has no obligation to put themselves in danger to help a human if there is a present and dangerous threat.

[LiarGG]

I actually don't think we disagree on the first part. The stipulation that "Not catching it if you have a chance is the same, it's just addressing the inaction part." was me addressing inaction as essentially just another side of the same coin. What I meant by that is that inaction, same as action, are just a choice the silicon can make, but the law is meant to prevent them from doing so. It cannot set in motion the chain of reactions leading to a human being harmed AND, when it spots such a reaction chain already happening, it is compelled to intervene and stop it so that no harm happens at the end. Sorry if I was unclear about that.

If the AI runs in to try to save the human, (who probably isnt even a human any more, a corpse isnt a human) and dies, its in violation of the first law. I would argue a borg has no obligation to put themselves in danger to help a human if there is a present and dangerous threat.

To address your second point, of course corpses aren't human and a borg is under no obligation to risk its life for one, perhaps unless he can do so safely under a law 2 order. My point there was more about addressing law 1 in light of law 3 and a difference between the borg being damaged and being destroyed. The way our laws are structured the borg is compelled to risk itself being harmed up until it risks immediate death in order to save a human that is dying. Once he is just a corpse, that internal conflict ends, but up until that point merely getting damaged is not enough for the cyborg to stop trying to prevent harm.

Actually, since stuns exist, I would probably grant a potential reasoning of "The killer had a flash/ions" as a valid reason to not engage, because in that case a death of the borg is all but guaranteed on a single misstep. At that point I'd bite the bullet on the same reasoning as I illustated in the SM example of "I ain't competent enough to go there and survive a fight against a flash."

To better illustrate my point of damage vs destruction, let's instead imagine an atmosian in a modsuit, who gets stuck in a lit burnchamber, surrounded by fierce plasma fire, and his oxygen tank runs out. Let's say he managed to call out for help before falling into soft-crit and a med-borg arrives on the scene while the man is still suffocating. Now I'd say that the borg is obligated to go into the burn chamber to fetch the dying man, even tho it will get some fire damage (iirc medborgs still get fire ticks, correct me if I'm wrong). The reasoning for this is that the risk of borgs destruction is next to none in this example, it will just get some fire damage.

I hope we can at least agree on this example, that in such a situation the silicon would be harmed, but it would be nowehere near risking its non-existence, which is the requirement for law 3 to override law 1 (well, not override, but for the AI to make a decision on which is more important, cuz it's comparing one human harm to another form of a human harm).

Actually this whole segment is a bit off topic because at this point we are going over law 1 internal conflict of "which harm is more important", which was not the original question. I just hope I have sufficiently demonstrated that there is a difference between silicon being harmed and silicon being non-existent and some applications of this logic regarding law 2 and 3

[tostah]

To better illustrate my point of damage vs destruction, let's instead imagine an atmosian in a modsuit, who gets stuck in a lit burnchamber, surrounded by fierce plasma fire, and his oxygen tank runs out. Let's say he managed to call out for help before falling into soft-crit and a med-borg arrives on the scene while the man is still suffocating. Now I'd say that the borg is obligated to go into the burn chamber to fetch the dying man, even tho it will get some fire damage (iirc medborgs still get fire ticks, correct me if I'm wrong). The reasoning for this is that the risk of borgs destruction is next to none in this example, it will just get some fire damage.

I hope we can at least agree on this example, that in such a situation the silicon would be harmed, but it would be nowehere near risking its non-existence, which is the requirement for law 3 to override law 1 (well, not override, but for the AI to make a decision on which is more important, cuz it's comparing one human harm to another form of a human harm).

I dont agree with this example. The borg is not compelled to step in and assist. That atmosian is straight up chilling in a firerpoof suit and is not currently being harmed. Even if they were, the borg is not obligated to step in to prevent their death if there is a chance of their deactivation.

What if the ai gets locked in? What if they are unable to escape? What if the damage was significantly higher than they thought?

Risking their life to save one person puts everyone in danger. An AI cannot prioritize one persons life over the safety of the entire crew. Now we're getting into the parts of AI in which Asimov himself struggled with. Which is more important, the AI putting humans in harms way by its actions (putting itself in a dangerous environment) or its inaction in allowing a person to die? We could avoid this by not treating the AIs existence as similar to a humans. We should avoid this scenario at all costs.

After defining things purely logicaly, I do see my logical fallacy. Turning this into a hypothetical syllogism gives us;

A always causes B, and C can cause A, therefore C can cause B. Define A as nonexistence, B as human harm, and C as damage.

This is a slippery slope fallacy. Damage does not always lead to nonexistence. It would be incorrect to say 'Damage will always lead to nonexistence', but correct to say 'Damage could lead to nonexistence'.

This possibility, even if it is a slippery slope fallacy, must be considered by an AI, as they cannot allow themselves to die via their inaction.

[LiarGG]

Eh, was hoping the atmos example is gonna work better because the only real way for a borg to get stuck there would be the APC dying while they are inside of the chamber, so the risk for any low-moderately competent silicon is basically non-existent AND, as I said, the atmosian was out of air, so no intervention would 100% lead to him suffocating and dying. Meaning this situation was comparing inaction that will definitely lead to a human dying and an action that will definitely hurt the silicon, but almost certainly won't destroy them. I'd say that in this scenario it is very safe to assume that non-existence is not on the table for a silicon with any level of competency and therefore not an issue with law 3.

On the note of AI having a direct loophole to be considered under law 1 being a bad thing, while pragmatically I'd agree in the real world, this is a sacrifice we have for the sake of gameplay, so that AI cannot just be ordered to self-destruct while having to oblige at any cost. It's a trade-off between AIs being able to do more than we'd really want of them, was this to be real life, and silicon roles being fun to play. Or less frustrating at least.

[tostah]

Eh, was hoping the atmos example is gonna work better because the only real way for a borg to get stuck there would be the APC dying while they are inside of the chamber, so the risk for any low-moderately competent silicon is basically non-existent AND, as I said, the atmosian was out of air, so no intervention would 100% lead to him suffocating and dying. Meaning this situation was comparing inaction that will definitely lead to a human dying and an action that will definitely hurt the silicon, but almost certainly won't destroy them. I'd say that in this scenario it is very safe to assume that non-existence is not on the table for a silicon with any level of competency and therefore not an issue with law 3.

On the note of AI having a direct loophole to be considered under law 1 being a bad thing, while pragmatically I'd agree in the real world, this is a sacrifice we have for the sake of gameplay, so that AI cannot just be ordered to self-destruct while having to oblige at any cost. It's a trade-off between AIs being able to do more than we'd really want of them, was this to be real life, and silicon roles being fun to play. Or less frustrating at least.

Youre absolutely right the chance is low, and a competent ai/borg would never get stuck. The possibility is there, and it provides the borg an excuse to not do something. These 'what ifs' give an AI the ability to subvert its laws.

If the entire concern is a human telling the ai to kill itself, just add metaprotections. Or, add a part to the second law, saying they can refuse orders which are for them to self-destruct or commit sucide in any way. That would be less flawed than what we have now.

What we're doing is allowing the 3rd law to affect the 1st law. This really shouldn't happen.

[LiarGG]

Youre absolutely right the chance is low, and a competent ai/borg would never get stuck. The possibility is there, and it provides the borg an excuse to not do something. These 'what ifs' give an AI the ability to subvert its laws.

If the entire concern is a human telling the ai to kill itself, just add metaprotections. Or, add a part to the second law, saying they can refuse orders which are for them to self-destruct or commit sucide in any way. That would be less flawed than what we have now.

What we're doing is allowing the 3rd law to affect the 1st law. This really shouldn't happen.

I suppose limiting what has to be written in the policy was kinda Timber's point when he was writing it. Actually don't remember whether we got Asimov+ or new silipol first to compare which influenced which.
Hell, might ask him at some point about his reasonings for what was and wasn't included.

That being said since asimov, like every lawset ever, is formed from natural language, it will always be up to interpretation. What we are looking for in our silicons (through my point of view anyway) is that the interpretation is 1. resonable and 2. consistent (within itself. New round with new interpretation is chill, just no double standards within the same round)

I suppose our biggest disconnect is your take on "immediacy" of a harm. Or not even your take, your approach that AI can argue about what immediacy is ad absurdum. Just like law 2 orders have a guiding policy that's concerned with immediacy, so does law 1 priority within itself, this being the policy clause:
2. Stopping any immediate harm takes priority over stopping any future harm.

My point is that in a situation where the borg is outside of the chamber and the atmosian is dying inside, at that precise moment there is a chain of events that's gonna lead to the atmosian suffocating, unless the borg intervenes.

The decision on the borg's part in that moment is either inaction, which will lead to the human dying, or joining this chain of events. Now the atmosian is already being harmed by suffocation, the harm to him is immediate in every sense of the word and no harm preceeds it. For the borg to refuse to go in, prioritising his own life, would be prioritising preventing harm that would be done at the time of his death, which is very much "prioritising future harm", which is in conflict with the policy.

In light of this policy, I'd say that silicons are already prevented from making the argument you are making. And if the silicon wishes to argue about immediacy, I'd say it's being unreasonable and just looking for hooks to play in bad faith. It's a fun intellectual exercise tho.

[Not-Dorsidarf]

This point is just to illustrate that AIs and borgs CAN perform an act that's basically a sabotage, if they are reasonably certain that the harm part of it (again explosion, pressure, shock, radiation and suffocation) can be averted. And decline such offer if they are unsure of this possibility. It's just a good rule of thumb to disregard these orders in case this would actually lead to harm, but still fully on the AI to evaluate.

I'm going to be very annoying and focus in on this moment, but if I'm not mistaken the SM can be brought back from virtually any state the AI can reasonably put it into without physical sabotage, as long as a competent engineer or two catches it fairly quickly.



I also kinda think the interpretation of "conflict must be an immediate conflict, not a potential future one" being used in this argument wrt law 2 orders vs future harm kinda has a problem not faced yet, because it means the policy immediately following that one doesn't really work. The entire justification for AIs not letting randoms into dangerous restricted areas like ordnance and the armory is that they are allowed to assume that doing so poses a Law 1 threat - aka, potential harm. By this definition, the AI still has to let them in because it's only a threatened violation of Law 1, not an actual one.

It also... kinda compels the AI to let a blood-covered HOS with a laser gun set to kill into the room with the human he's been hunting and previously shooting, because the conflict with Law 1 is only potential (He might switch back to non-lethal mode, he might pull out a baton, the human might baton him before he can harm them, and so on), which would in all other cases be considered a ridiculous interpretation, and making it the unambiguously correct one by policy seems... off. Silicons need to be able to consider non-certain events in the near future for law 1 to work.

I dont think any of this really effects anyone's argument however so this was probbably a waste of time.

[LiarGG]

I'm going to be very annoying and focus in on this moment, but if I'm not mistaken the SM can be brought back from virtually any state the AI can reasonably put it into without physical sabotage, as long as a competent engineer or two catches it fairly quickly.

Not annoying in the slightest
The SM certainly can be saved at any point, but it needs competent technicians (or at least a competent cyborg). AI itself cannot save most sabotages, because it has no way of dealing with clogs and overpressurized chambers. My stance is that an AI should not sabotage the SM unless it is reasonably certain it can save it, meaning having a competent borg or at least knowing exactly how to instruct someone in how to fix the issue.

I also kinda think the interpretation of "conflict must be an immediate conflict, not a potential future one" being used in this argument wrt law 2 orders vs future harm kinda has a problem not faced yet, because it means the policy immediately following that one doesn't really work. The entire justification for AIs not letting randoms into dangerous restricted areas like ordnance and the armory is that they are allowed to assume that doing so poses a Law 1 threat - aka, potential harm. By this definition, the AI still has to let them in because it's only a threatened violation of Law 1, not an actual one.

It also... kinda compels the AI to let a blood-covered HOS with a laser gun set to kill into the room with the human he's been hunting and previously shooting, because the conflict with Law 1 is only potential (He might switch back to non-lethal mode, he might pull out a baton, the human might baton him before he can harm them, and so on), which would in all other cases be considered a ridiculous interpretation, and making it the unambiguously correct one by policy seems... off. Silicons need to be able to consider non-certain events in the near future for law 1 to work.

I'd say the issue of access is addressed within' policy as well. The section about silicons and access states that opening doors without immediate human harm threat is not harmful, and then it ammends that statement by listing places that can be considered harmful. Imma be honest my original reading of the policy from way back then let me to believe that the act of letting someone into dangerous areas is a harmful act in and of itself. Reading this as an act of potential harm is making it fully unusable, because then it would not concern itself with law 1 at all. And considering it as harmful, but in the future, not imminent, would just mean that you can open it if it was, for example, for the purpose of someone escaping harm - which is gonna be my position from now on, cuz that makes way more sense. That being said, considering action chains, AI can still deny access under law 2, because it would again start an action chain that the AI knows will lead to human harm (because of the policy).

HoS situation is the same in that if the AI can establish that a harm is about to happen, it can deny access.

[Not-Dorsidarf]

I'd say the issue of access is addressed within' policy as well. The section about silicons and access states that opening doors without immediate human harm threat is not harmful, and then it ammends that statement by listing places that can be considered harmful. Imma be honest my original reading of the policy from way back then let me to believe that the act of letting someone into dangerous areas is a harmful act in and of itself. Reading this as an act of potential harm is making it fully unusable, because then it would not concern itself with law 1 at all. And considering it as harmful, but in the future, not imminent, would just mean that you can open it if it was, for example, for the purpose of someone escaping harm - which is gonna be my position from now on, cuz that makes way more sense. That being said, considering action chains, AI can still deny access under law 2, because it would again start an action chain that the AI knows will lead to human harm (because of the policy).

HoS situation is the same in that if the AI can establish that a harm is about to happen, it can deny access.

See, I don't get the action chain angle here - because it makes sense with mechanical systems like dropping a piano or setting the SM to explode, but seems to fall apart with inherently unpredictable situations (aka virtually any time other players are involved), where the X -> Y -> Z chain becomes a lot harder to justify without encompassing situations way outside the intent of what we want AIs to consider.

[LiarGG]

See, I don't get the action chain angle here - because it makes sense with mechanical systems like dropping a piano or setting the SM to explode, but seems to fall apart with inherently unpredictable situations (aka virtually any time other players are involved), where the X -> Y -> Z chain becomes a lot harder to justify without encompassing situations way outside the intent of what we want AIs to consider.

Sure, I mean since we are playing a multiplayer RP game, most chains can have 3rd party involvement that can prevent or cause harm. I'd say that the best rule of thumb is "If an action chain would lead to human harm with no other person intevening, the AI is obligated to break such a chain." This will work for most "combined chains", because as long as the AI can break the chain, it shouldn't rely on 3rd party to do so, because them failing would lead to human harm by inaction. And in a situation where a chain is not leading to human harm, AI is not obligated to do anything until a 3rd party makes it so that human harm is on the table. Then the former applies - break it.

All this is still mostly just about AI not doing overtly risky shit like sabotaging SM without ensuring it won't lead to human harm.

Now for the HoS example, it will really boil down to purely AI interpretation imho. If the HoS is chasing a person and asks to open a door for him, it's up to the AI to gauge the likeliness of that action resulting in harm. Since this is a grey zone, both answers can be true most of the time. It's about the AI establishing an intent. For example, if the HoS has earlier said that they are gonna kill that person, that establishes an intent and the AI shouldn't open for him. The HoS can then say something like "I was joking" or "I have changed my mind" and at that point, it's once again a situation that's fully on the AI to interpret. Both option would be then considered valid in my eyes.

I'd say that most law conflicts are gonna be quite surface level and what I am mostly trying to establish here is a viewpoint that, in my experience, works well for interpreting laws, policy and orders in a way that's both logically sound and safe (in a way that's not gonna get you bwoinked). most of the times you ain't gonna get into trouble by opening a door for a secoff, unless they are like activelly shooting at someone. On the other hand refusing most orders with shoddy reasoning, not caring about human harm, or shit like sabotaging SMs are likely to get some eyebrows raised, because it's typically not what we want from our AIs and the policy and laws are meant to reflect that.

My overall standpoint is that as long as the AI is acting in good faith, it's gonna be generally safe.