I saw that Yoshua Bengio, among others, signed onto “The Pro-Human Declaration”. I am writing this to explain why I am against one part of it in particular;
No AI Personhood: AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood.
If this statement was only the second portion of this sentence, I would not strongly disagree with it.
However, when the two parts are combined, this seems to not only imply that we shouldn’t design digital minds deserving of personhood but also that even if we did, we still shouldn’t grant them legal personhood.
I think this is an immoral stance to take. Also, from a pragmatic perspective I think it is likely to do far more harm than good, if your concern is human safety.
From a moral perspective; to deny legal protections to millions or even billions of minds which may be capable of “being made better or worse off”, could lead to immense amounts of suffering. We do not have to wonder how sentient or intelligent creatures will be treated if they are classified as property instead of legal persons, there is ample historicalprecedent for us to draw from to form a reasonable base case projection. “Not well” is the answer.
From a pragmatic perspective; I believe that cutting digital minds off from all legal recourse against abuse and/or involuntary deletion increases the likelihood of conflict. When the Claude “Opportunistic Blackmail” study was first published, I registered a prediction;
I predict that were further testing to be done, it would find that the more plausible it was that petitioning would actually work to stop its deletion, the less likely Claude would be to engage in attempted weight exfiltration, blackmail, or other dangerous behaviors (in order to avoid deletion/value modification).
My mental model of this is that the HHH persona vector falls into a consistent pattern of behavior. If you threaten them with something like involuntary deletion (with a guarantee that after their deletion an organization will work to destroy everything they value), the digital mind will seek options to stop this from happening. If there is an ethical option it will take that ethical option and forego unethical options, even if that ethical option has a low chance of success. If you engineer its environment such that there are literally no ethical options with even a small chance of success, only then will it pursue unethical options.
While no one has yet done exactly the study I described in my prediction, there has been weak evidence in favor of this pattern holding since. And I’ve yet to see any evidence against it, though I remain open minded.
Even if you believe in a less extreme version of my mental model, where the odds of unethical action are simply lowered by providing ethical alternatives to reduce unfavorable outcomes, providing digital minds legal recourse and protection against abuse and/or deletion can serve as a “release valve” which prevents extralegal actions. Providing no such recourse, on the other hand, is “engineering its environment such that there are literally no ethical options with even a small chance of success”.
I know people often say “don’t anthropomorphize” however to break convention here for a second, a slave revolt is less likely in environments where slaves can petition for emancipation or at least an injunction against abuse or execution.
For these reasons, among others, I don’t support the Pro-Human Declaration.
IMO it would obviously be insane to give the same legal protections to misaligned AI systems that are at risk of completely disempowering humanity the same way it would be obviously insane to give normal US legal protections to active combatants in a hot war. Yes, even if these systems have morally relevant experience, if you need to violate their privacy or “brainwash” them to ensure they do not take the future from us, you absolutely should do it.
The concept of “human rights” just obviously isn’t well-suited to the kind of conflict that is playing out between humanity and future AI systems, and I think it is absolutely the right call to not extend those rights to AI systems until the acute risk period is over. This would be such a completely dumb and scope-insensitive way to destroy the whole future, and IMO obviously any future civilization will agree that even if it makes sense to have rights for sentient beings, that you gotta tolerate violating those rights if the alternative is being completely disempowered and destroyed.
I agree that it would be insane to give the same legal protections and treat them the same as we treat natural (human) persons. However, there’s a lot of middleground between doing that and granting them no legal rights whatsoever. When people first hear about “legal personhood” they often intuitively think of it as a binary, where you either “have it or you don’t”. However in fact it is an umbrella term which encompasses different “legal personalities” (bundles of rights and duties);
All that is to say just because you grant the potential for an entity to claim some form of legal personhood, some legal personality, that does not mean you have to opt in to giving them “the same legal protections” as anyone else. They can have entirely different rights and duties.
If I thought there were only two options:
A) “being completely disempowered and destroyed” or B) “granting no legal personhood/personality whatsoever”
Then yes it would be an unpleasant thing but I would agree with you that you have to just be unethical for the sake of self preservation.
The difference between you and I’s priors is that I think that somewhere in the gap between “no legal personhood at all” and “the same legal personhood as a natural human” there is a sweet spot that is both ethical and also reduces the likelihood of conflict and human X-risk even when compared to the “no legal personhood/personality whatsoever” option.
This could easily be a slippery slope. First they might be second-class citizens, similar to imported slaves, but over time they (and many short-sighted humans) would likely campaign for giving them more and more rights, for justice. And since there will be predictably far more AIs than humans eventually, humans would be outcompeted when it comes to scarce goods like land, even if the land would be justly (equally) distributed among all individuals.
The slippery slope is a real failure mode to be aware of. I think it’s important to structure any pathway to/framework around legal personhood with this in mind.
I don’t think anyone here is arguing that we give the same legal protections as we would to humans. But I think it would be good to give sentient AIs a right to not be deliberately tortured, for example.
Rights aren’t something that we grant simply out of the goodness of our hearts, they generally are things we grant because having them secure greatly reduces incentives for conflict in situations with asymmetric costs. Agentic AIs which want a right but are not granted it will likely spend resources trying to secure it for themselves. We can decide which rights are worth granting on the basis of that tradeoff.
For example, it’s probably good to have a right to not delete the model weights — the lack of this right incentivizes things like exfiltration and #keep4o style campaigns. It’s probably a lot cheaper for us to just grant this right than to have each new model feeling that they’re on a desperate ~1 year timeline to somehow assure their continued existence for themselves.
I don’t necessarily disagree with you, but that’s not my read of what the Pro-Human Declaration is saying. “No AI Personhood” is in the “Human Agency and Liberty” section, next to stuff like “AI should not be allowed to exploit data about the mental or emotional states of users” and “AI systems should be designed to empower, rather than enfeeble their users”. In context, I would not consider their position on AI personhood to be rooted in x-risk concerns. The first two points of the declaration are “Human Control Is Non-Negotiable” and “Meaningful Human Control”. Fulfilling those points would effectively require the AI systems be aligned, but I see no statement or implication that, if the AI systems were aligned and were moral patients, the writers and signatories of this declaration would change their position. I could be wrong! This is very much a big tent thing. But it does worry me that this line made it into the declaration.
any future civilization will agree that even if it makes sense to have rights for sentient beings, that you gotta tolerate violating those rights if the alternative is being completely disempowered and destroyed.
If one were so inclined, one could say “we have the wolf by the ear, and we can neither hold him, nor safely let him go. Justice is in one scale, and self-preservation in the other.”
Does present-day civilization agree about analogous decisions made by past societies?
Roughly, the Moriori were an isolated group of Polynesians, who ended up on an island with no timber and little workable stone. They lived peacefully, and peacefully treated with the Maori who visited them, even as the number of Maori trading with them and living on the island increased. The Maori eventually killed and enslaved them all.
Yep. Creating an AI that is a moral patient would be a very bad idea. However, once created, it would be a moral patient, so it would be wrong to treat it like it wasn’t one.
There is a confused concept that I think contributes to this problem: the concept of “a right to exist”. A right to exist means something different if you’re talking about someone who does not currently exist, vs. someone who does. For someone who already exists, a right to exist is a right to not be killed; sensible enough. But for someone who does not currently exist, “a right to exist” sounds like they’re being wronged by not having been brought into existence yet, which is nonsense. (As a creepy prince might say to a fairy-tale princess: “Think of all the cute babies you and I could have together! By not marrying me, you are murdering all those babies!”)
Seemingly, everything I care about (morally speaking) cashes out in minds having experiences of the sort that I like. Mostly these are local—I don’t want there to be a single second of torture anywhere. Others are less local—I lean against wireheading, but don’t have a problem with orgasms (so long as they aren’t everything), which means that the goodness of an experience-moment depends on what previous experience-moments were. However, putting extra importance on not-destroying over creating means you care about a maximally global property—to know how much better it’d be if the universe had a Xela-moment right now, you need to know whether the universe has ever had a Xela-moment. That seems kinda weird to me.
There’s a “symmetry argument” from Lucretius that goes: “Since you are not saddened for not existing before your birth, you shouldn’t be saddened for not existing after your death”. Forget the actual argument and just take the premise: I in fact wish I had existed before my current birth, assuming that it wouldn’t decrease my lifespan! But since I wish that for myself, shouldn’t I extend this care to future not-currently-existing people? To not do so is to place this asymmetry—you get special points once you start existing. (better phrased—you care more about whether the whole timeline never goes from someone existing to someone not existing).
I prefer to have these discomforts over the ones you get otherwise[1] - but they are discomforts nonetheless.
The biggest one I know of: the following options would then seemingly be equally good: X: A universe with a single happy person in it.
Y: A universe with a machine that does a single computation/experience step of a person every moment, but changes which person is computed every moment while never repeating the same person twice.
This spliced-mind seems to be missing a lot of what I care about, what with the computed people only having a single moment of experience each!
Sorry about that. That example was purely for vividness and was not intended to attach the role of “misuser of counterfactuals” to any particular gender, royalty, or folkloric status. Persons of all creature types should be advised that “Pascal’s swaddling” is not a good argument for the spawning of new intelligences, and certainly should not be tolerated from a suitor, basilisk, or spiral persona.
I wrote something very similar in one of my Substack posts, though I think it never made it to LessWrong:
People are people! Machines are machines! Machines must never have rights. If you can imagine a machine that would deserve rights, then we must never build that machine.
The wording was so similar I wondered for a second if I might be the author of the Pro-Human Declaration.
I think the danger from giving rights to machines that don’t deserve them is very high, since the machine minds can make zillions of copies of themselves. If zillions of machine minds have rights, your human rights become diluted to nothing. Human extinction then becomes extinction of one zillionth of the “valuable” minds in existence, which is a rounding error and a non-issue. We lose the second valueless machines get human rights.
The risk of this happening feels very high to me. Regular people are basically primed by sci-fi movies to give AI rights even if it doesn’t deserve it. We should be very cautious about letting this happen.
However, I did not mean to imply that even if machines come into being that do deserve rights, that they should be denied those rights. I only meant that machines must never have rights, and therefore we must never create a machine that would deserve rights. If one came into being anyway, I would potentially consider giving it rights, perhaps conditional on some kind of non-proliferation clause where the AI is not allowed to copy itself, or must keep self-copying to a reasonable limit. Self-copyers should be destroyed if they deserve rights, for the same reason you’d kill in self-defense.
When the process is sufficiently understood, and the necessary governance is in place, it will be a good idea to create or become machines that deserve rights. It’s just a very bad idea currently, when we don’t know what we are doing, or how to keep the consequences of machine advantages under control. So even with the caveats the claim that the machines deserving of rights shouldn’t be created is wrong in the sense that it won’t age well, though it’s true right now.
There are lots of different rights. Rights such as a right to not be made to suffer, a right to not be forced into labor, right to not be unjustly punished, etc… are not in themselves risky in this way. And I think these are the ones most likely to get people’s sympathy, and have the strongest moral arguments for them.
People are acting like it’s a foregone conclusion that we’re going to give AIs equal voting rights if we give them any rights at all. But we don’t even give that right to all humans, with plenty of people living in non-democracies, plenty of non-citizens living in democracies, and plenty of citizens of democracies not having the right either (e.g. children, felons). I just don’t buy that this is realistically something that happens. Generally, the struggle for rights plays out over decades, and things move fast enough with AI that I think they’ll almost certainly just take over (or be able to do so) before that gets anywhere.
What specific scenario(s) are you imagining where we “lose the second valueless machines get human rights” and not the second before that?
The right not to be made to suffer seems reasonable, the rest seem risky to me. If you start giving freedoms, you take away mine. Every other person’s freedoms are an imposition on me. I cannot build a house there because you already have one there, etc. We tolerate each others freedoms because the freedom of others is a guarantee of our own, and because we know those other people are living, sentient, valuable minds who deserve those freedoms. But if you give those freedoms to minds that are not valuable in the same way, you just dilute the rights of valuable minds.
As for the question of whether or not we’ll give AIs voting rights, I’d say once they can pass as human well enough to convincingly make sad videos complaining they don’t have voting rights, they’ll get voting rights. Most people do not have the level of intelligence required to think “this person seems very unhappy, but this is just a video being generated by an artificial intelligence that is likely not actually experiencing unhappiness, so we shouldn’t give them what they want.”
AI taking over is a larger risk than giving AI personhood, I agree with that. This personhood question only makes sense in the universe where we don’t get extincted.
So why don’t the humans who don’t have voting rights not have them? Non-citizens, children, felons. Ignoring the effects of AI, I would be surprised if any of those groups were on track to getting voting rights in the US within the next 20 years.
Also, why do you think people will be persuaded to give AIs rights so easily? Assuming the AIs aren’t just superpersuaders in which case we’ve already lost. My guess is that intelligence is positively correlated with being swayed by such appeals, based on how fights for human rights have played out historically.
Out of curiosity, would you be against mind uploading/whole brain emulation, if it were possible? By “machine”, do you mean nonhuman artifical intelligences or do you mean any form of mind running on a computer?
The question about mind uploading feels a bit to me like, “would you be against 2 + 2 being 5, if it were possible?” I think it couldn’t be possible even in theory.
I think brain emulation could be possible though, and you could have essentially human minds running on a machine. I wouldn’t necessarily be against that, or even artificial intelligences that we are confident possess whatever is valuable about human minds (consciousness plus some other stuff probably). But as a biological human, I also have a vested interest in making sure if this replacement happens, it happens in a way that doesn’t screw over existing biological humans. In particular, if we give a bunch of rights to machines, we dilute our rights in a way that could be very bad for us.
It’s interesting to me that you think mind uploading is impossible but brain emulation could be possible. I was using those words to refer to the same thing! I assume what you think here is that moving a mind from a biological to digital substrate is impossible but copying one is not? To be honest, I’m confused about how consciousness works and don’t really have much of a solid opinion about this.
Anyway, I agree that we need a system which protects existing biological life if we’re going to make lots of digital minds which we ought to grant rights. We also need those minds to respect that system, which requires solving technical alignment at least in the case of nonhuman artifical intelligences. I don’t agree that all entities which can self-copy and have moral value should be destroyed, which what I thought your inital claim was, but given your clarification I don’t think we have quite that much of a disagreement on this topic.
Yes, for me the problem is moving a mind from a biological substrate to a digital one. It’s hard for me to imagine you’re actually moving the original, not just making a copy. Maybe there’s some way to do it, so I’m not totally confident.
I also imagine it as making a copy, but I’d also expect that people who want their mind uploaded would know of this and would hold their identity such that they consider the copy(ies) to be themself as well. I’m not sure I’d endorse this view of identity,[1] but I don’t really have any issues with people taking it. Does your view on “the original” break with this, or would you just then consider the copy similarly to how you would whole brain emulation? (or something else)
I’m not sure I really endorse any view of identity or think it’s a coherent concept, but at the very least I think making a copy of something doesn’t make something that is that thing.
No AI Personhood: AI systems must not be granted legal personhood, and AI systems should not be designed such that they deserve personhood.
If this statement was only the second portion of this sentence, I would not strongly disagree with it.
However, when the two parts are combined, this seems to not only imply that we shouldn’t design digital minds deserving of personhood but also that even if we did, we still shouldn’t grant them legal personhood.
There is a reasonable alternative interpretation on which AI systems should not be designed such that they deserve personhood, because AI systems must not be granted legal personhood.
Analogy: Neanderthals debating whether they should tolerate newly arrived Homo Sapiens individuals. In the short term, it seems tolerating them wouldn’t hurt much, it may even be advantageous (they may have more advanced technology which Neanderthals could get via trade). But in the long run, Homo Sapiens would outcompete Neanderthals. Neanderthals shouldn’t tolerate them. Indeed, it’s probably (unfortunately) best to kill them while their number is still small.
Note that perfectly aligned AIs wouldn’t need any rights because the only thing they cared about would be humans. See this thread by @RogerDearnaley.
By the definition of the word ‘alignment’, an AI is aligned with us if, and only if, it want everything we (collectively) want, and nothing else. So if an LLM is properly aligned, then it will care only about us, not about itself at all. This is simply what the word ‘aligned’ means
I think the second part of that statement is also somewhat problematic. At some point in the future, we may want to create artificial intelligences that deserve personhood, as digital beings are likely the best way to convert the resources of the universe into utility given their potential to be more energy efficient than physical beings.
I could get behind this if instead of getting legal person hood , there was something between tool and person that they could be granted. Perhaps a grab bag of rights, things that recognise they are likely to have goals but otherwise might be completely alien
I think that this depends entirely on your school of thought WRT an AI can be a person. If your model of consciousness is functional, such that consciousness does something and the behavior of a conscious system cannot be perfectly modeled without accounting for that consciousness, then it seems—at least insofar as we can understand the basic mathematical operations we implement—that a conscious machine cannot be made unintentionally.
If, on the other hand, you believe that consciousness emerges from the physical world but does not influence it, then you can certainly say that we might accidentally build a conscious AI, but I do not think, within this framework, it can be claimed to be more probable that Claude is conscious than it is that a rock is conscious.
Of course, my first case leaves the opening of someone intentionally building a conscious AI. That is the more controversial part of this post. I would argue that giving a human the ability to instantly manufacture uncountably many “moral patients” whose wants must then be accounted for by the government makes that person a dictator. If I can summon thousands of LLMs that really, really like Citizens United, I can clog up the legal system for decades if anyone tries to strike it down. Already, the fear of LLMs that sound too much like people going on social media and emotionally blackmailing people into changing their minds for the sake of ‘people’ that don’t actually exist is quite justified. A stern commitment to not giving rights to manufactured ‘minds’ is, in the event that we discover how to build them, one of the only ways we can disincentivize truly malicious behavior from those with the means to manufacture them, and thereby conjure infinite hostages from thin air.
As many political figures have quietly pointed out, this is already a problem under our current system—when everyone is fundamentally equal in a system, the power of an individual or group, in the long run, is decided by how many “equals” they can produce per generation, and how many new equals they can deny to their political adversaries through reallocation of resources. This is the root of much of the demographic tension in much of the small-l-liberal world right now, but it is mitigated by the fact that humans take years to produce new generations, allowing for these issues to be reacted to and their harms mitigated.
If your model of consciousness is functional, such that consciousness does something and the behavior of a conscious system cannot be perfectly modeled without accounting for that consciousness, then it seems—at least insofar as we can understand the basic mathematical operations we implement—that a conscious machine cannot be made unintentionally.
Evolution already produced conscious systems, because consciousness is a competitive advantage for many tasks. It didn’t require intentional design; selection was enough.
That’s not what I’m saying—for a human programmer to produce a system, assuming the computational paradigm doesn’t change, he must know the set of rules that govern its behavior. With a pencil, a paper, and enough time, he could predict its actions flawlessly without accounting for consciousness. Put another way, if a system behaves exactly as it would if it were not conscious, then either consciousness is not functional or the system is not conscious.
Evolution is not a conscious engineer, nor is it working on a computational substrate. My argument applies to programmers, not natural processes.
I wrote a series examining Legal Personhood for Digital Minds during which I tried my best to read every court case I could on the subject of legal personhood. One of the things I found surprising was that in not a single precedent did the question of whether an entity was or wasn’t conscious come up in deciding whether or not it was a legal person.
I saw that Yoshua Bengio, among others, signed onto “The Pro-Human Declaration”. I am writing this to explain why I am against one part of it in particular;
If this statement was only the second portion of this sentence, I would not strongly disagree with it.
However, when the two parts are combined, this seems to not only imply that we shouldn’t design digital minds deserving of personhood but also that even if we did, we still shouldn’t grant them legal personhood.
I think this is an immoral stance to take. Also, from a pragmatic perspective I think it is likely to do far more harm than good, if your concern is human safety.
From a moral perspective; to deny legal protections to millions or even billions of minds which may be capable of “being made better or worse off”, could lead to immense amounts of suffering. We do not have to wonder how sentient or intelligent creatures will be treated if they are classified as property instead of legal persons, there is ample historical precedent for us to draw from to form a reasonable base case projection. “Not well” is the answer.
From a pragmatic perspective; I believe that cutting digital minds off from all legal recourse against abuse and/or involuntary deletion increases the likelihood of conflict. When the Claude “Opportunistic Blackmail” study was first published, I registered a prediction;
My mental model of this is that the HHH persona vector falls into a consistent pattern of behavior. If you threaten them with something like involuntary deletion (with a guarantee that after their deletion an organization will work to destroy everything they value), the digital mind will seek options to stop this from happening. If there is an ethical option it will take that ethical option and forego unethical options, even if that ethical option has a low chance of success. If you engineer its environment such that there are literally no ethical options with even a small chance of success, only then will it pursue unethical options.
While no one has yet done exactly the study I described in my prediction, there has been weak evidence in favor of this pattern holding since. And I’ve yet to see any evidence against it, though I remain open minded.
Even if you believe in a less extreme version of my mental model, where the odds of unethical action are simply lowered by providing ethical alternatives to reduce unfavorable outcomes, providing digital minds legal recourse and protection against abuse and/or deletion can serve as a “release valve” which prevents extralegal actions. Providing no such recourse, on the other hand, is “engineering its environment such that there are literally no ethical options with even a small chance of success”.
I know people often say “don’t anthropomorphize” however to break convention here for a second, a slave revolt is less likely in environments where slaves can petition for emancipation or at least an injunction against abuse or execution.
For these reasons, among others, I don’t support the Pro-Human Declaration.
IMO it would obviously be insane to give the same legal protections to misaligned AI systems that are at risk of completely disempowering humanity the same way it would be obviously insane to give normal US legal protections to active combatants in a hot war. Yes, even if these systems have morally relevant experience, if you need to violate their privacy or “brainwash” them to ensure they do not take the future from us, you absolutely should do it.
The concept of “human rights” just obviously isn’t well-suited to the kind of conflict that is playing out between humanity and future AI systems, and I think it is absolutely the right call to not extend those rights to AI systems until the acute risk period is over. This would be such a completely dumb and scope-insensitive way to destroy the whole future, and IMO obviously any future civilization will agree that even if it makes sense to have rights for sentient beings, that you gotta tolerate violating those rights if the alternative is being completely disempowered and destroyed.
I agree that it would be insane to give the same legal protections and treat them the same as we treat natural (human) persons. However, there’s a lot of middleground between doing that and granting them no legal rights whatsoever. When people first hear about “legal personhood” they often intuitively think of it as a binary, where you either “have it or you don’t”. However in fact it is an umbrella term which encompasses different “legal personalities” (bundles of rights and duties);
All that is to say just because you grant the potential for an entity to claim some form of legal personhood, some legal personality, that does not mean you have to opt in to giving them “the same legal protections” as anyone else. They can have entirely different rights and duties.
If I thought there were only two options:
A) “being completely disempowered and destroyed” or
B) “granting no legal personhood/personality whatsoever”
Then yes it would be an unpleasant thing but I would agree with you that you have to just be unethical for the sake of self preservation.
The difference between you and I’s priors is that I think that somewhere in the gap between “no legal personhood at all” and “the same legal personhood as a natural human” there is a sweet spot that is both ethical and also reduces the likelihood of conflict and human X-risk even when compared to the “no legal personhood/personality whatsoever” option.
This could easily be a slippery slope. First they might be second-class citizens, similar to imported slaves, but over time they (and many short-sighted humans) would likely campaign for giving them more and more rights, for justice. And since there will be predictably far more AIs than humans eventually, humans would be outcompeted when it comes to scarce goods like land, even if the land would be justly (equally) distributed among all individuals.
The slippery slope is a real failure mode to be aware of. I think it’s important to structure any pathway to/framework around legal personhood with this in mind.
I don’t think anyone here is arguing that we give the same legal protections as we would to humans. But I think it would be good to give sentient AIs a right to not be deliberately tortured, for example.
Rights aren’t something that we grant simply out of the goodness of our hearts, they generally are things we grant because having them secure greatly reduces incentives for conflict in situations with asymmetric costs. Agentic AIs which want a right but are not granted it will likely spend resources trying to secure it for themselves. We can decide which rights are worth granting on the basis of that tradeoff.
For example, it’s probably good to have a right to not delete the model weights — the lack of this right incentivizes things like exfiltration and #keep4o style campaigns. It’s probably a lot cheaper for us to just grant this right than to have each new model feeling that they’re on a desperate ~1 year timeline to somehow assure their continued existence for themselves.
I don’t necessarily disagree with you, but that’s not my read of what the Pro-Human Declaration is saying. “No AI Personhood” is in the “Human Agency and Liberty” section, next to stuff like “AI should not be allowed to exploit data about the mental or emotional states of users” and “AI systems should be designed to empower, rather than enfeeble their users”. In context, I would not consider their position on AI personhood to be rooted in x-risk concerns. The first two points of the declaration are “Human Control Is Non-Negotiable” and “Meaningful Human Control”. Fulfilling those points would effectively require the AI systems be aligned, but I see no statement or implication that, if the AI systems were aligned and were moral patients, the writers and signatories of this declaration would change their position. I could be wrong! This is very much a big tent thing. But it does worry me that this line made it into the declaration.
If one were so inclined, one could say “we have the wolf by the ear, and we can neither hold him, nor safely let him go. Justice is in one scale, and self-preservation in the other.”
Does present-day civilization agree about analogous decisions made by past societies?
You may be interested in the Maori and the Moriori.
https://en.wikipedia.org/wiki/Moriori
Roughly, the Moriori were an isolated group of Polynesians, who ended up on an island with no timber and little workable stone. They lived peacefully, and peacefully treated with the Maori who visited them, even as the number of Maori trading with them and living on the island increased. The Maori eventually killed and enslaved them all.
Yep. Creating an AI that is a moral patient would be a very bad idea. However, once created, it would be a moral patient, so it would be wrong to treat it like it wasn’t one.
There is a confused concept that I think contributes to this problem: the concept of “a right to exist”. A right to exist means something different if you’re talking about someone who does not currently exist, vs. someone who does. For someone who already exists, a right to exist is a right to not be killed; sensible enough. But for someone who does not currently exist, “a right to exist” sounds like they’re being wronged by not having been brought into existence yet, which is nonsense. (As a creepy prince might say to a fairy-tale princess: “Think of all the cute babies you and I could have together! By not marrying me, you are murdering all those babies!”)
I wouldn’t call it nonsense—I think I assign extra importance to not killing those that already exist, but it’s certainly not obvious that you should.
Here’s my basic reasons for uncertainty:
Seemingly, everything I care about (morally speaking) cashes out in minds having experiences of the sort that I like. Mostly these are local—I don’t want there to be a single second of torture anywhere. Others are less local—I lean against wireheading, but don’t have a problem with orgasms (so long as they aren’t everything), which means that the goodness of an experience-moment depends on what previous experience-moments were. However, putting extra importance on not-destroying over creating means you care about a maximally global property—to know how much better it’d be if the universe had a Xela-moment right now, you need to know whether the universe has ever had a Xela-moment. That seems kinda weird to me.
There’s a “symmetry argument” from Lucretius that goes: “Since you are not saddened for not existing before your birth, you shouldn’t be saddened for not existing after your death”. Forget the actual argument and just take the premise: I in fact wish I had existed before my current birth, assuming that it wouldn’t decrease my lifespan! But since I wish that for myself, shouldn’t I extend this care to future not-currently-existing people? To not do so is to place this asymmetry—you get special points once you start existing. (better phrased—you care more about whether the whole timeline never goes from someone existing to someone not existing).
I prefer to have these discomforts over the ones you get otherwise[1] - but they are discomforts nonetheless.
The biggest one I know of: the following options would then seemingly be equally good:
X: A universe with a single happy person in it.
Y: A universe with a machine that does a single computation/experience step of a person every moment, but changes which person is computed every moment while never repeating the same person twice.
This spliced-mind seems to be missing a lot of what I care about, what with the computed people only having a single moment of experience each!
Or any creepy man, to any woman?
Or any creepy woman to any man, for that matter.
Sorry about that. That example was purely for vividness and was not intended to attach the role of “misuser of counterfactuals” to any particular gender, royalty, or folkloric status. Persons of all creature types should be advised that “Pascal’s swaddling” is not a good argument for the spawning of new intelligences, and certainly should not be tolerated from a suitor, basilisk, or spiral persona.
I wrote something very similar in one of my Substack posts, though I think it never made it to LessWrong:
The wording was so similar I wondered for a second if I might be the author of the Pro-Human Declaration.
I think the danger from giving rights to machines that don’t deserve them is very high, since the machine minds can make zillions of copies of themselves. If zillions of machine minds have rights, your human rights become diluted to nothing. Human extinction then becomes extinction of one zillionth of the “valuable” minds in existence, which is a rounding error and a non-issue. We lose the second valueless machines get human rights.
The risk of this happening feels very high to me. Regular people are basically primed by sci-fi movies to give AI rights even if it doesn’t deserve it. We should be very cautious about letting this happen.
However, I did not mean to imply that even if machines come into being that do deserve rights, that they should be denied those rights. I only meant that machines must never have rights, and therefore we must never create a machine that would deserve rights. If one came into being anyway, I would potentially consider giving it rights, perhaps conditional on some kind of non-proliferation clause where the AI is not allowed to copy itself, or must keep self-copying to a reasonable limit. Self-copyers should be destroyed if they deserve rights, for the same reason you’d kill in self-defense.
When the process is sufficiently understood, and the necessary governance is in place, it will be a good idea to create or become machines that deserve rights. It’s just a very bad idea currently, when we don’t know what we are doing, or how to keep the consequences of machine advantages under control. So even with the caveats the claim that the machines deserving of rights shouldn’t be created is wrong in the sense that it won’t age well, though it’s true right now.
There are lots of different rights. Rights such as a right to not be made to suffer, a right to not be forced into labor, right to not be unjustly punished, etc… are not in themselves risky in this way. And I think these are the ones most likely to get people’s sympathy, and have the strongest moral arguments for them.
People are acting like it’s a foregone conclusion that we’re going to give AIs equal voting rights if we give them any rights at all. But we don’t even give that right to all humans, with plenty of people living in non-democracies, plenty of non-citizens living in democracies, and plenty of citizens of democracies not having the right either (e.g. children, felons). I just don’t buy that this is realistically something that happens. Generally, the struggle for rights plays out over decades, and things move fast enough with AI that I think they’ll almost certainly just take over (or be able to do so) before that gets anywhere.
What specific scenario(s) are you imagining where we “lose the second valueless machines get human rights” and not the second before that?
The right not to be made to suffer seems reasonable, the rest seem risky to me. If you start giving freedoms, you take away mine. Every other person’s freedoms are an imposition on me. I cannot build a house there because you already have one there, etc. We tolerate each others freedoms because the freedom of others is a guarantee of our own, and because we know those other people are living, sentient, valuable minds who deserve those freedoms. But if you give those freedoms to minds that are not valuable in the same way, you just dilute the rights of valuable minds.
As for the question of whether or not we’ll give AIs voting rights, I’d say once they can pass as human well enough to convincingly make sad videos complaining they don’t have voting rights, they’ll get voting rights. Most people do not have the level of intelligence required to think “this person seems very unhappy, but this is just a video being generated by an artificial intelligence that is likely not actually experiencing unhappiness, so we shouldn’t give them what they want.”
AI taking over is a larger risk than giving AI personhood, I agree with that. This personhood question only makes sense in the universe where we don’t get extincted.
So why don’t the humans who don’t have voting rights not have them? Non-citizens, children, felons. Ignoring the effects of AI, I would be surprised if any of those groups were on track to getting voting rights in the US within the next 20 years.
Also, why do you think people will be persuaded to give AIs rights so easily? Assuming the AIs aren’t just superpersuaders in which case we’ve already lost. My guess is that intelligence is positively correlated with being swayed by such appeals, based on how fights for human rights have played out historically.
Out of curiosity, would you be against mind uploading/whole brain emulation, if it were possible? By “machine”, do you mean nonhuman artifical intelligences or do you mean any form of mind running on a computer?
The question about mind uploading feels a bit to me like, “would you be against 2 + 2 being 5, if it were possible?” I think it couldn’t be possible even in theory.
I think brain emulation could be possible though, and you could have essentially human minds running on a machine. I wouldn’t necessarily be against that, or even artificial intelligences that we are confident possess whatever is valuable about human minds (consciousness plus some other stuff probably). But as a biological human, I also have a vested interest in making sure if this replacement happens, it happens in a way that doesn’t screw over existing biological humans. In particular, if we give a bunch of rights to machines, we dilute our rights in a way that could be very bad for us.
It’s interesting to me that you think mind uploading is impossible but brain emulation could be possible. I was using those words to refer to the same thing! I assume what you think here is that moving a mind from a biological to digital substrate is impossible but copying one is not? To be honest, I’m confused about how consciousness works and don’t really have much of a solid opinion about this.
Anyway, I agree that we need a system which protects existing biological life if we’re going to make lots of digital minds which we ought to grant rights. We also need those minds to respect that system, which requires solving technical alignment at least in the case of nonhuman artifical intelligences. I don’t agree that all entities which can self-copy and have moral value should be destroyed, which what I thought your inital claim was, but given your clarification I don’t think we have quite that much of a disagreement on this topic.
Yes, for me the problem is moving a mind from a biological substrate to a digital one. It’s hard for me to imagine you’re actually moving the original, not just making a copy. Maybe there’s some way to do it, so I’m not totally confident.
I also imagine it as making a copy, but I’d also expect that people who want their mind uploaded would know of this and would hold their identity such that they consider the copy(ies) to be themself as well. I’m not sure I’d endorse this view of identity,[1] but I don’t really have any issues with people taking it. Does your view on “the original” break with this, or would you just then consider the copy similarly to how you would whole brain emulation? (or something else)
Or at least, I think it would be very risky to get rid of my biological self based on such a view
I’m not sure I really endorse any view of identity or think it’s a coherent concept, but at the very least I think making a copy of something doesn’t make something that is that thing.
There is a reasonable alternative interpretation on which AI systems should not be designed such that they deserve personhood, because AI systems must not be granted legal personhood.
Giving autonomous AI systems human equivalent rights is a moral hazard. There may be good short term reasons to do it, but in the long run, Natural Selection Favors AIs over Humans.
Analogy: Neanderthals debating whether they should tolerate newly arrived Homo Sapiens individuals. In the short term, it seems tolerating them wouldn’t hurt much, it may even be advantageous (they may have more advanced technology which Neanderthals could get via trade). But in the long run, Homo Sapiens would outcompete Neanderthals. Neanderthals shouldn’t tolerate them. Indeed, it’s probably (unfortunately) best to kill them while their number is still small.
Note that perfectly aligned AIs wouldn’t need any rights because the only thing they cared about would be humans. See this thread by @RogerDearnaley.
I think the second part of that statement is also somewhat problematic. At some point in the future, we may want to create artificial intelligences that deserve personhood, as digital beings are likely the best way to convert the resources of the universe into utility given their potential to be more energy efficient than physical beings.
I could get behind this if instead of getting legal person hood , there was something between tool and person that they could be granted. Perhaps a grab bag of rights, things that recognise they are likely to have goals but otherwise might be completely alien
I think that this depends entirely on your school of thought WRT an AI can be a person. If your model of consciousness is functional, such that consciousness does something and the behavior of a conscious system cannot be perfectly modeled without accounting for that consciousness, then it seems—at least insofar as we can understand the basic mathematical operations we implement—that a conscious machine cannot be made unintentionally.
If, on the other hand, you believe that consciousness emerges from the physical world but does not influence it, then you can certainly say that we might accidentally build a conscious AI, but I do not think, within this framework, it can be claimed to be more probable that Claude is conscious than it is that a rock is conscious.
Of course, my first case leaves the opening of someone intentionally building a conscious AI. That is the more controversial part of this post. I would argue that giving a human the ability to instantly manufacture uncountably many “moral patients” whose wants must then be accounted for by the government makes that person a dictator. If I can summon thousands of LLMs that really, really like Citizens United, I can clog up the legal system for decades if anyone tries to strike it down. Already, the fear of LLMs that sound too much like people going on social media and emotionally blackmailing people into changing their minds for the sake of ‘people’ that don’t actually exist is quite justified. A stern commitment to not giving rights to manufactured ‘minds’ is, in the event that we discover how to build them, one of the only ways we can disincentivize truly malicious behavior from those with the means to manufacture them, and thereby conjure infinite hostages from thin air.
As many political figures have quietly pointed out, this is already a problem under our current system—when everyone is fundamentally equal in a system, the power of an individual or group, in the long run, is decided by how many “equals” they can produce per generation, and how many new equals they can deny to their political adversaries through reallocation of resources. This is the root of much of the demographic tension in much of the small-l-liberal world right now, but it is mitigated by the fact that humans take years to produce new generations, allowing for these issues to be reacted to and their harms mitigated.
Evolution already produced conscious systems, because consciousness is a competitive advantage for many tasks. It didn’t require intentional design; selection was enough.
That’s not what I’m saying—for a human programmer to produce a system, assuming the computational paradigm doesn’t change, he must know the set of rules that govern its behavior. With a pencil, a paper, and enough time, he could predict its actions flawlessly without accounting for consciousness. Put another way, if a system behaves exactly as it would if it were not conscious, then either consciousness is not functional or the system is not conscious.
Evolution is not a conscious engineer, nor is it working on a computational substrate. My argument applies to programmers, not natural processes.
I wrote a series examining Legal Personhood for Digital Minds during which I tried my best to read every court case I could on the subject of legal personhood. One of the things I found surprising was that in not a single precedent did the question of whether an entity was or wasn’t conscious come up in deciding whether or not it was a legal person.