“Moreover so much more than what could exist does”
Why would that be?
Pure combinatorics. You could potentially have children with everyone you encounter. Now some of those are exceedingly unlikely, but even if 1⁄100 of them had a significant probability, that’s likely at least on order of magnitude or more than the people that you do wind up having kids with. For every potential coparent, there are a lot more children that you could have, but won’t. There are just too many factors that determine the outcome of a pregnancy. Again, we’re talking orders of magnitude more potential children than actual children. when we talk about all of the possible states of the world, versus the actual state of the world, the difference in orders of magnitude is simply astronomical.
Most things that could exist, don’t exist. There are far more possible worlds that have no Basilisk than ones that do. Now, you’re right to include how likely a particular potential world is, but even if we say in all worlds with AGI, humans are worse off, the likelihood of a Basilisk is vanishingly small, compared to all of the ways things could go wrong. Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule.
I don’t think that the nature of the torture matters. I think that I could think of a scenario where it would be worth enduring. It’s hard to balance torture against the welfare of others, but once we are in the billions, that feels pretty clear to me. The negative value of being tortured for 10,000 years can’t possibly be lower than the torture and deaths of billions of people. There is always a scenario where it is worth enduring. The risk is always finite.
But let’s take a step back, and presume that a Basilisk scenario is possible. What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
There is no reason to cooperate with something so horrible: it can’t be reasoned with nor negotiated with—causally or not.
It’s astronomically unlikely to happen; and if it did there is no value in cooperating. If you create it, then you are the monster, whether you were inspired by Rosco or not.
Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into.
So go back. Why is it unlikely that an ASI would reward those that help create it, rather than punish those that don’t? You dismissed angels, but this seems to me the far more-likely scenario. It’s basically the default, otherwise what’s the point of building them in the first place? Now that doesn’t mean the angel doesn’t kill us all too, but it doesn’t engage in all this torture causal trade nonsense.
I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence.
I’m just not about to change my life and become a supervillain henchman, but if some ASI slid into my DM’s and said, “Yo, Jason. I’ll give you $2 million dollars to write some software for me. He’s proof I’m sincere,” I’d at least listen and ask about the benefits package.
There is no thought trap, other than what you create for yourself.
Let’s consider a functionally equivalent ASI scenario to a Basilisk. Let’s call it Jason’s Hobgoblin. An ASI comes into existence decides to ultra-torture everyone, with maybe some small chance of a reprieve based on whether it likes you or not. No acausal trade. It just sees who helped it exist, and chooses to make some of them its pets. The Hobgoblin takes up a bunch of space of the Basilisk futures.
Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?
I think that a chance for a reward from a Basilisk is equally inscrutable. You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to.
Why cooperate when you have no idea what the actual effect will be? Well, other than the damage you might do as its henchman. And the cost to your mental health as you go around the anxiety loop.
Even if you believe the Basilisk is a likely future, there’s no reason to cooperate with it, or give it further thought than any other possible future.
If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives?
If the Basilisk is just a story, then is not worth worrying about. If the Basilisk is just one of any number possible futures, then there is no reason to give it special attention. If the Basilisk is the future, then there is no point is cooperating with it.
Pure combinatorics. You could potentially have children with everyone you encounter. Now some of those are exceedingly unlikely, but even if 1⁄100 of them had a significant probability, that’s likely at least on order of magnitude or more than the people that you do wind up having kids with. For every potential coparent, there are a lot more children that you could have, but won’t. There are just too many factors that determine the outcome of a pregnancy. Again, we’re talking orders of magnitude more potential children than actual children. when we talk about all of the possible states of the world, versus the actual state of the world, the difference in orders of magnitude is simply astronomical.
Most things that could exist, don’t exist. There are far more possible worlds that have no Basilisk than ones that do. Now, you’re right to include how likely a particular potential world is, but even if we say in all worlds with AGI, humans are worse off, the likelihood of a Basilisk is vanishingly small, compared to all of the ways things could go wrong. Even in the worlds where there is a Basilisk, given variation in population, and AGI timelines, the chance of you being targeted is minuscule.
I don’t think that the nature of the torture matters. I think that I could think of a scenario where it would be worth enduring. It’s hard to balance torture against the welfare of others, but once we are in the billions, that feels pretty clear to me. The negative value of being tortured for 10,000 years can’t possibly be lower than the torture and deaths of billions of people. There is always a scenario where it is worth enduring. The risk is always finite.
But let’s take a step back, and presume that a Basilisk scenario is possible. What harms are you willing to do to make sure it is created? Would you create such a monster? Even in a world where a Basilisk is inevitable, what harms would you cause? Would they be worth it? What if it decides to just go ahead and torture you anyway?
There is no reason to cooperate with something so horrible: it can’t be reasoned with nor negotiated with—causally or not.
It’s astronomically unlikely to happen; and if it did there is no value in cooperating. If you create it, then you are the monster, whether you were inspired by Rosco or not.
Rosco’s Basilisk is an intellectual trap of your own making. It’s delusion: a rationalization of the irrational. It’s not worth thinking of, and especially not worth buying into.
It might make a good novel though.
Comment withdrawn.
So go back. Why is it unlikely that an ASI would reward those that help create it, rather than punish those that don’t? You dismissed angels, but this seems to me the far more-likely scenario. It’s basically the default, otherwise what’s the point of building them in the first place? Now that doesn’t mean the angel doesn’t kill us all too, but it doesn’t engage in all this torture causal trade nonsense.
I just don’t understand why this particular scenario seems likely. Especially since it’s unlikely to work, given how most people don’t give it much credence.
I’m just not about to change my life and become a supervillain henchman, but if some ASI slid into my DM’s and said, “Yo, Jason. I’ll give you $2 million dollars to write some software for me. He’s proof I’m sincere,” I’d at least listen and ask about the benefits package.
There is no thought trap, other than what you create for yourself.
Let’s consider a functionally equivalent ASI scenario to a Basilisk. Let’s call it Jason’s Hobgoblin. An ASI comes into existence decides to ultra-torture everyone, with maybe some small chance of a reprieve based on whether it likes you or not. No acausal trade. It just sees who helped it exist, and chooses to make some of them its pets. The Hobgoblin takes up a bunch of space of the Basilisk futures.
Now, do you change your life to try to get on its good side before it even exists? I don’t think so: it’s crazy. How can you really understand why the Hobgoblin likes you, or does what it does?
I think that a chance for a reward from a Basilisk is equally inscrutable. You’re already considering cooperating with it, so it doesn’t have to actually cooperate with you. You have no way of knowing if it will cooperate with you it’s not actually incentivized to.
Why cooperate when you have no idea what the actual effect will be? Well, other than the damage you might do as its henchman. And the cost to your mental health as you go around the anxiety loop.
Even if you believe the Basilisk is a likely future, there’s no reason to cooperate with it, or give it further thought than any other possible future.
If the Hobgoblin splits the Basilisk probability space, then it’s it likely that there are other similar scenarios that do as well. Maybe an Angel is a Hobgoblin in disguise? Doesn’t this lead us back to the Basilisk not being a particularly likely possible future given all of the alternatives?
If the Basilisk is just a story, then is not worth worrying about. If the Basilisk is just one of any number possible futures, then there is no reason to give it special attention. If the Basilisk is the future, then there is no point is cooperating with it.
Comment withdrawn.
Well, those are my best arguments. I hope I’ve been helpful in some way.
Thanks for engaging with my question.