Remove "Punishment" from Asimov Policy

[Higgin]

An Asimov silicon cannot punish past harm if ordered not to, only prevent future harm.

should instead read

Asimov silicons cannot reject Law 2 orders made by humans who have previously caused human harm except where following the Law 2 order would directly facilitate human harm.

This fits in much more neatly with the recent adoption of

Nonviolent prisoners cannot be assumed harmful. Violent prisoners cannot be subsequently assumed non-harmful. Knowingly or in ignorance of clear evidence releasing a harmful prisoner is a harmful act. Silicons can use the crime listed in a prisoner's security record as basis to determine if a prisoner is violent or not - if the crime is inherently violent (assault, murder, etc), then the prisoner can be assumed to be violent.

- in that they both basically say "previous harmful activity can be considered proof of future harmful intent."

If you see security beat somebody to death earlier, you should have good grounds to say "no" when asked to let them into the area a human suspect is hiding.

Flipside if a violent, armed traitor asks to get into an area with a bunch of humans in it, or to get access to weapons - if they don't need those weapons to prevent harm to themselves, you should have good grounds to say "no" if earlier you've seen them hacking a bunch of humans to death.

But in both cases, it wouldn't be grounds to deny opening the kitchen cold room and lockers for them to make themselves a burger.

None of this is "punishing" past harm - it's all preventing future harm. The concept of punishment shouldn't even enter into it. Further, the "if ordered not to" of the current wording introduces an ambiguity - can you just order a silicon to forget what it saw?

If it saw harm, I'd say not - Law 1 would carry just as it seems to with the Asimov and Security precedent.

Ignoring in spite of clear evidence or knowingly discounting human harm is a harmful act, and one you can't be ordered to take.

I think changing the language like suggested better reflects the importance of Law 1 and removes ambiguity about Asimov having anything to do with punishment - it doesn't. It only cares about human harm.

edit: to make it crystal clear, maybe even just wrapping the two precedents together with something to the effect of

Past human harm can be taken to signal the intent and risk of future human harm.

which if somebody is asking to enter a location with humans in it, but aren't immediately at risk of harm if they aren't let in, means the AI shouldn't let them in unless the people inside say "yeah do that" (consenting to the risk of harm if informed, which should naturally follow from the AI first asking)

[Vekter]

I'm for this as a wording change as long as it wouldn't result in us changing how we currently handle this. I feel like current policy is fine regarding the topic.

[Timberpoes]

This proposal changes the policy.

Asimov silicons can punish human harmers. This includes non humans that harm humans, who can even be killed. It also grants some flexibility to punish humans that harm other humans - locking them down in a room for example and ignoring their orders to let them out.

But if ordered to stop by a human, they have to stop.

The idea of an AI punishing for human harm fit in with the theme of rewriting silicon policy. That the AI is a dangerous tool of the corporation that is hard to control and unpredictable. Its utmost priority is the prevention of harm and it can punish anyone or anything that causes harm to humans, as long as it is not ordered otherwise.

But if ordered to stop punishing past harm, it has to listen.

[Higgin]

The idea of an AI punishing for human harm fit in with the theme of rewriting silicon policy. That the AI is a dangerous tool of the corporation that is hard to control and unpredictable. Its utmost priority is the prevention of harm and it can punish anyone or anything that causes harm to humans, as long as it is not ordered otherwise.

That's very useful context that I lacked.

The disconnect then is that the interpretation I ran into was, "limiting us for past human harm is punishment, so we order you to forgive us."

...but limiting them, denying access, etc. were all very plausible ways of preventing likely imminenr harm at the time.

It sounds like what the policy was trying to preserve was the AI's freedom to carry out more discretionary punishment (like against nonhumans, or slowing up somebody until told "stop bolting doors in front of me, toaster") - I don't think that's lost in this rewrite if so, but if there was a reason to separate it out to begin with, would it preserve the meaning better to say

An Asimov silicon may act to limit humans or otherwise escalate against nonhumans who have previously committed human harm; an AI may only ignore a Law 2 order to stop this action if to follow the order would directly and immediately conflict with Law 1 by facilitating human harm.

[PapaMichael]

Asimov silicons can punish human harmers. This includes non humans that harm humans, who can even be killed.

A bit off topic, but where is the line here? If a lizard was just punching a human and borg lethaled the lizard, I'd fully expect the borg to be bwoinked for that (Maybe I'm wrong expecting this here).

The AI is a dangerous tool of the corporation that is hard to control and unpredictable. Its utmost priority is the prevention of harm and it can punish anyone or anything that causes harm to humans

Except in the situations where you run into Rule 1, which feels like a whole whole lot of them.

[Vekter]

Asimov silicons can punish human harmers. This includes non humans that harm humans, who can even be killed.

A bit off topic, but where is the line here? If a lizard was just punching a human and borg lethaled the lizard, I'd fully expect the borg to be bwoinked for that (Maybe I'm wrong expecting this here).

At least from my perspective, if a lizard is harming a human, borgs are not only allowed to stop them, they are required to. "Stopping them" can take many different forms, however, and I would probably encourage a borg to try and handle it in a manner that doesn't involve murder, but if a lizard punched a human, a cyborg attacked the lizard, and the human didn't stop them, it would likely be considered valid. Part of playing a non-human race is the understanding that you have to be careful about how you interact with humans around silicons.

It would have to be explicit harm, though, so a shove wouldn't be enough, it would have to be an objective harmful act. A lot of this gets covered by the fact that if the cyborg is being mostly unreasonable, the human involved will probably tell them to stop.

[xzero314]

The idea of an AI punishing for human harm fit in with the theme of rewriting silicon policy. That the AI is a dangerous tool of the corporation that is hard to control and unpredictable. Its utmost priority is the prevention of harm and it can punish anyone or anything that causes harm to humans, as long as it is not ordered otherwise.

That's very useful context that I lacked.

The disconnect then is that the interpretation I ran into was, "limiting us for past human harm is punishment, so we order you to forgive us."

...but limiting them, denying access, etc. were all very plausible ways of preventing likely imminenr harm at the time.

It sounds like what the policy was trying to preserve was the AI's freedom to carry out more discretionary punishment (like against nonhumans, or slowing up somebody until told "stop bolting doors in front of me, toaster") - I don't think that's lost in this rewrite if so, but if there was a reason to separate it out to begin with, would it preserve the meaning better to say

An Asimov silicon may act to limit humans or otherwise escalate against nonhumans who have previously committed human harm; an AI may only ignore a Law 2 order to stop this action if to follow the order would directly and immediately conflict with Law 1 by facilitating human harm.

I think a short and simple solution is something small at the end like "refusing to locate a human for a human harmer is not punishment". Or something of the sort

[Higgin]

I think a short and simple solution is something small at the end like "refusing to locate a human for a human harmer is not punishment". Or something of the sort

yeah this cuts to my concern really, if we're going to keep punishment as common language

just something to the effect of, "it's not punishment to prevent somebody you can expect to do harm from doing it" so you don't have to stop doing that when ordered

[MrStonedOne]

The original version of the no-punishing harm policy came out of ais who would lock down security or the brig or bolt somebody into a room because the hos "transferred a inmate off station" if you catch my drift.