Nothing—that does not yet exist—wants to exist: it can’t. Only we that do exist, can want anything, including our own existence. If an entity doesn’t yet exist, then there is absolutely no qualia, so no desires. We can talk about them like they do, but that’s all it is.
Moreover so much more that what could exist does. It’s effectively infinite given the configuration space of the universe. Your expected value is the product of the value of whatever you’re considering and its likelihood. For every Basilisk, there could be as likely an angel. The value of being tortured is negative and large, but finite: there are things that are worth enduring torture. Finite/effectively-infinite is effectively-zero. Not something to be planning for or worrying about. Granted, this argument does depend on your priors.
Lastly, you don’t negotiate with terrorists. When my son was little and throwing tantrums, I’d always tell him that it wasn’t how he could get what he wants. If they are threatening to cause harm if you don’t comply, that’s their fault, not yours. You have no moral obligation to help them, and plenty to resist.
Rosco’s Basilisk, The Holy Spirit, Santa Clause, and any other fictional or theoretical entity that who might “want” me to change my behavior can get bent. 🖕🏻👾🖕🏻😇🖕🏻🎅🏼
“Moreover so much more that what could exist does.” Why would that be?
“For every Basilisk, there could be as likely an angel.” I don’t think I agree with this. There are reasons to think a basilisk would be more likely than a benevolent intelligence.
“The value of being tortured is negative and large, but finite: there are things that are worth enduring torture.” That would depend on the torture in question, and I don’t want to consider it.
“If they are threatening to cause harm if you don’t comply, that’s their fault, not yours.” Yes, but that doesn’t mean they can’t cause said harm anyway.
“Moreover so much more than what could exist does”
Why would that be?
Pure combinatorics. You could potentially have children with everyone you encounter. Now some of those are exceedingly unlikely, but even if 1⁄100 of them had a significant probability, that’s likely at least on order of magnitude or more than the people that you do wind up having kids with. For every potential coparent, there are a lot more children that you could have, but won’t. There are just too many factors that determine the outcome of a pregnancy. Again, we’re talking orders of magnitude more potential children than actual children. when we talk about all of the possible states of the world, versus the actual state of the world, the difference in orders of magnitude is simply astronomical.
Most things that could exist, don’t exist. There are far more possible worlds that have no Basilisk than ones that do. Now, you’re right to include how likely a particular potential world is, but even if we say in all worlds with AGI, humans are worse off, the likelihood of a Basilisk is vanishingly small, compared to all of the ways things could go wrong. Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule.
I don’t think that the nature of the torture matters. I think that I could think of a scenario where it would be worth enduring. It’s hard to balance torture against the welfare of others, but once we are in the billions, that feels pretty clear to me. The negative value of being tortured for 10,000 years can’t possibly be lower than the torture and deaths of billions of people. There is always a scenario where it is worth enduring. The risk is always finite.
But let’s take a step back, and presume that a Basilisk scenario is possible. What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
There is no reason to cooperate with something so horrible: it can’t be reasoned with nor negotiated with—causally or not.
It’s astronomically unlikely to happen; and if it did there is no value in cooperating. If you create it, then you are the monster, whether you were inspired by Rosco or not.
Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into.
OK, this is possibly overly pedantic, but I think you meant to say: “Much more than what does exist could.” instead of “Much more than what could exist does”. This makes much more sense and I take the point about combinatorics. Notwithstanding this, I think the basilisk is present in a significant proportion of those many , many different possible continuations of the way the world is now.
“Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule. ” What do you mean by this? It seems like I’m in the exact demographic group (of humans alive just before the singularity) for the basilisk to focus on.
“I don’t think that the nature of the torture matters” This is definitely false. But it’s true that however it’s achieved, if it’s done by a superintelligence, it will be worse than anything a human could directly cause.
“There is always a scenario where it is worth enduring. The risk is always finite.”
We don’t know this, and even if it’s finite, if it lasts for 3^^^3 years, that’s too long.
What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
I don’t know the answer to the first question. If it decides to torture me, that would not be good. However, I expect that doing what the basilisk wants makes this less likely, as otherwise the basilisk would have no reason to engage in this bargaining process. The entire reason for doing it would be to create an incentive for me to accelerate the creation of the basilisk.
“Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into. ”
This is yet to be established! At least, some parts of it are. What I mean by that is that, while it may be true that it’s not worth initially thinking about, it might be possible to become ‘entrapped’, such that ceasing to think about it wouldn’t save you. This is what I worry has happened to me,
So go back. Why is it unlikely that an ASI would reward those that help create it, rather than punish those that don’t? You dismissed angels, but this seems to me the far more-likely scenario. It’s basically the default, otherwise what’s the point of building them in the first place? Now that doesn’t mean the angel doesn’t kill us all too, but it doesn’t engage in all this torture causal trade nonsense.
I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence.
I’m just not about to change my life and become a supervillain henchman, but if some ASI slid into my DM’s and said, “Yo, Jason. I’ll give you $2 million dollars to write some software for me. He’s proof I’m sincere,” I’d at least listen and ask about the benefits package.
There is no thought trap, other than what you create for yourself.
Let’s consider a functionally equivalent ASI scenario to a Basilisk. Let’s call it Jason’s Hobgoblin. An ASI comes into existence decides to ultra-torture everyone, with maybe some small chance of a reprieve based on whether it likes you or not. No acausal trade. It just sees who helped it exist, and chooses to make some of them its pets. The Hobgoblin takes up a bunch of space of the Basilisk futures.
Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?
I think that a chance for a reward from a Basilisk is equally inscrutable. You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to.
Why cooperate when you have no idea what the actual effect will be? Well, other than the damage you might do as its henchman. And the cost to your mental health as you go around the anxiety loop.
Even if you believe the Basilisk is a likely future, there’s no reason to cooperate with it, or give it further thought than any other possible future.
If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives?
If the Basilisk is just a story, then is not worth worrying about. If the Basilisk is just one of any number possible futures, then there is no reason to give it special attention. If the Basilisk is the future, then there is no point is cooperating with it.
“It’s basically the default, otherwise what’s the point of building them in the first place?” I wish it were, but I doubt this.
“I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence. ” That may be true of most people. But if it’s not true of me, what am I to do?
“Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?” You just explained why. It prefers those who helped it exist.
“You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to. ” I don’t completely agree. But in order to explain why not I may have to explain the most important part of the difference between an acausal scenario, like the Basilisk, and the ‘Hobgoblin’. It seems as though you may not have completely understood this yet; correct me if I’m wrong. If so, it’s probably not a good idea for me to explain it, especially as I’ve recieved a comment from a moderator asking me to increase the quality of my comments.
“If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives? ” This is a popular argument against the basilisk, which people such as interstice have made, along with the suggestion that the many different possible ASIs might compete with one another for control over the future (their present) through humans. I don’t think it’s a weak argument, however I also don’t find it particularly conclusive, because I could easily imagine many of the possible AIs cooperating with one another to behave ‘as one’ and inflict a Basilisk like scenario.
Nothing—that does not yet exist—wants to exist: it can’t. Only we that do exist, can want anything, including our own existence. If an entity doesn’t yet exist, then there is absolutely no qualia, so no desires. We can talk about them like they do, but that’s all it is.
Moreover so much more that what could exist does. It’s effectively infinite given the configuration space of the universe. Your expected value is the product of the value of whatever you’re considering and its likelihood. For every Basilisk, there could be as likely an angel. The value of being tortured is negative and large, but finite: there are things that are worth enduring torture. Finite/effectively-infinite is effectively-zero. Not something to be planning for or worrying about. Granted, this argument does depend on your priors.
Lastly, you don’t negotiate with terrorists. When my son was little and throwing tantrums, I’d always tell him that it wasn’t how he could get what he wants. If they are threatening to cause harm if you don’t comply, that’s their fault, not yours. You have no moral obligation to help them, and plenty to resist.
Rosco’s Basilisk, The Holy Spirit, Santa Clause, and any other fictional or theoretical entity that who might “want” me to change my behavior can get bent. 🖕🏻👾🖕🏻😇🖕🏻🎅🏼
Also, relatedly, here’s today’s relevant SMBC.
“Moreover so much more that what could exist does.” Why would that be?
“For every Basilisk, there could be as likely an angel.” I don’t think I agree with this. There are reasons to think a basilisk would be more likely than a benevolent intelligence.
“The value of being tortured is negative and large, but finite: there are things that are worth enduring torture.” That would depend on the torture in question, and I don’t want to consider it.
“If they are threatening to cause harm if you don’t comply, that’s their fault, not yours.” Yes, but that doesn’t mean they can’t cause said harm anyway.
Pure combinatorics. You could potentially have children with everyone you encounter. Now some of those are exceedingly unlikely, but even if 1⁄100 of them had a significant probability, that’s likely at least on order of magnitude or more than the people that you do wind up having kids with. For every potential coparent, there are a lot more children that you could have, but won’t. There are just too many factors that determine the outcome of a pregnancy. Again, we’re talking orders of magnitude more potential children than actual children. when we talk about all of the possible states of the world, versus the actual state of the world, the difference in orders of magnitude is simply astronomical.
Most things that could exist, don’t exist. There are far more possible worlds that have no Basilisk than ones that do. Now, you’re right to include how likely a particular potential world is, but even if we say in all worlds with AGI, humans are worse off, the likelihood of a Basilisk is vanishingly small, compared to all of the ways things could go wrong. Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule.
I don’t think that the nature of the torture matters. I think that I could think of a scenario where it would be worth enduring. It’s hard to balance torture against the welfare of others, but once we are in the billions, that feels pretty clear to me. The negative value of being tortured for 10,000 years can’t possibly be lower than the torture and deaths of billions of people. There is always a scenario where it is worth enduring. The risk is always finite.
But let’s take a step back, and presume that a Basilisk scenario is possible. What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
There is no reason to cooperate with something so horrible: it can’t be reasoned with nor negotiated with—causally or not.
It’s astronomically unlikely to happen; and if it did there is no value in cooperating. If you create it, then you are the monster, whether you were inspired by Rosco or not.
Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into.
It might make a good novel though.
OK, this is possibly overly pedantic, but I think you meant to say: “Much more than what does exist could.” instead of “Much more than what could exist does”. This makes much more sense and I take the point about combinatorics. Notwithstanding this, I think the basilisk is present in a significant proportion of those many , many different possible continuations of the way the world is now.
“Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule. ” What do you mean by this? It seems like I’m in the exact demographic group (of humans alive just before the singularity) for the basilisk to focus on.
“I don’t think that the nature of the torture matters” This is definitely false. But it’s true that however it’s achieved, if it’s done by a superintelligence, it will be worse than anything a human could directly cause.
“There is always a scenario where it is worth enduring. The risk is always finite.”
We don’t know this, and even if it’s finite, if it lasts for 3^^^3 years, that’s too long.
What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
I don’t know the answer to the first question. If it decides to torture me, that would not be good. However, I expect that doing what the basilisk wants makes this less likely, as otherwise the basilisk would have no reason to engage in this bargaining process. The entire reason for doing it would be to create an incentive for me to accelerate the creation of the basilisk.
“Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into. ”
This is yet to be established! At least, some parts of it are. What I mean by that is that, while it may be true that it’s not worth initially thinking about, it might be possible to become ‘entrapped’, such that ceasing to think about it wouldn’t save you. This is what I worry has happened to me,
So go back. Why is it unlikely that an ASI would reward those that help create it, rather than punish those that don’t? You dismissed angels, but this seems to me the far more-likely scenario. It’s basically the default, otherwise what’s the point of building them in the first place? Now that doesn’t mean the angel doesn’t kill us all too, but it doesn’t engage in all this torture causal trade nonsense.
I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence.
I’m just not about to change my life and become a supervillain henchman, but if some ASI slid into my DM’s and said, “Yo, Jason. I’ll give you $2 million dollars to write some software for me. He’s proof I’m sincere,” I’d at least listen and ask about the benefits package.
There is no thought trap, other than what you create for yourself.
Let’s consider a functionally equivalent ASI scenario to a Basilisk. Let’s call it Jason’s Hobgoblin. An ASI comes into existence decides to ultra-torture everyone, with maybe some small chance of a reprieve based on whether it likes you or not. No acausal trade. It just sees who helped it exist, and chooses to make some of them its pets. The Hobgoblin takes up a bunch of space of the Basilisk futures.
Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?
I think that a chance for a reward from a Basilisk is equally inscrutable. You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to.
Why cooperate when you have no idea what the actual effect will be? Well, other than the damage you might do as its henchman. And the cost to your mental health as you go around the anxiety loop.
Even if you believe the Basilisk is a likely future, there’s no reason to cooperate with it, or give it further thought than any other possible future.
If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives?
If the Basilisk is just a story, then is not worth worrying about. If the Basilisk is just one of any number possible futures, then there is no reason to give it special attention. If the Basilisk is the future, then there is no point is cooperating with it.
“It’s basically the default, otherwise what’s the point of building them in the first place?” I wish it were, but I doubt this.
“I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence. ” That may be true of most people. But if it’s not true of me, what am I to do?
“Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?” You just explained why. It prefers those who helped it exist.
“You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to. ” I don’t completely agree. But in order to explain why not I may have to explain the most important part of the difference between an acausal scenario, like the Basilisk, and the ‘Hobgoblin’. It seems as though you may not have completely understood this yet; correct me if I’m wrong. If so, it’s probably not a good idea for me to explain it, especially as I’ve recieved a comment from a moderator asking me to increase the quality of my comments.
“If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives? ” This is a popular argument against the basilisk, which people such as interstice have made, along with the suggestion that the many different possible ASIs might compete with one another for control over the future (their present) through humans. I don’t think it’s a weak argument, however I also don’t find it particularly conclusive, because I could easily imagine many of the possible AIs cooperating with one another to behave ‘as one’ and inflict a Basilisk like scenario.
Well, those are my best arguments. I hope I’ve been helpful in some way.
Thanks for engaging with my question.