a) You have to decide on a moral framework that can be explained in detail, to anyone.
b) It will be implemented worldwide tomorrow.
c) Tomorrow, every single human on Earth, including you and everyone you know, will also have their lives randomly swapped with someone else.
This is a fridge horror scenario. The more I consider it, the creepier and more horrifying it gets.
So 8 billion people are going to find themselves in someone else’s body of random age and sex, a large fraction unable to speak with the complete strangers around them, with no idea what has happened to their family, their existing moral framework ripped out of their minds and replaced with some external one that doesn’t fit with their preexisting experiences, with society completely dysfunctional and only one person who knows what’s going on?
Regardless of details of whatever moral framework is chosen, that’s an immense evil right there. If fewer than a billion people die before food infrastructure is restored, it would be a miracle.
Don’t worry JB. When a future SI and its totally-in-control principals impose the perfect framework, all will be well. We only need to find the magic bullet for making strong AI and then AI will solve everything.
As to the infrastructure concerns, don’t worry about that either. Alignment researchers and engineers of the highest pedigree keep telling us to just focus on sound technical solutions for better AI. Coordination challenges are apparently trivial. AI will do it for us at no or low cost.
This is a fridge horror scenario. The more I consider it, the creepier and more horrifying it gets.
So 8 billion people are going to find themselves in someone else’s body of random age and sex, a large fraction unable to speak with the complete strangers around them, with no idea what has happened to their family, their existing moral framework ripped out of their minds and replaced with some external one that doesn’t fit with their preexisting experiences, with society completely dysfunctional and only one person who knows what’s going on?
Regardless of details of whatever moral framework is chosen, that’s an immense evil right there. If fewer than a billion people die before food infrastructure is restored, it would be a miracle.
Don’t worry JB. When a future SI and its totally-in-control principals impose the perfect framework, all will be well. We only need to find the magic bullet for making strong AI and then AI will solve everything.
As to the infrastructure concerns, don’t worry about that either. Alignment researchers and engineers of the highest pedigree keep telling us to just focus on sound technical solutions for better AI. Coordination challenges are apparently trivial. AI will do it for us at no or low cost.
/s