Sentience is one of the basic goods. If the sysop is non-sentient, then whatever computronium is used in the sysop is, WRT sentience, wasted.
If we suppose that intelligences have a power-law distribution, and the sysop is the one at the top, we’ll find that it uses up something around 20% to 50% of the accessible universe’s computronium.
That would be a natural (as in “expected in nature”) distribution. But since the sysop needs to stay in charge, it will probably destroy any other AIs who reach the “second tier” of intelligence. So it will more likely have something like 70% − 90% of the universe’s computronium.
Also, in this post-human world, there aren’t large penalties for individuality. That is: In today’s world, you can’t add up 3 chimpanzee brains and get human-level intelligence. In the world of AIs, you probably can. This means that, to stay on top, the sysop will always need to reserve a majority of the universe’s computronium for itself. Otherwise, the rest of the universe can gang up on it.
So creating a non-sentient sysop means cutting the amount of sentient life you can support by at least half.
Sentience is one of the basic goods. If the sysop is non-sentient, then whatever computronium is used in the sysop is, WRT sentience, wasted.
Not necessarily. It depends on what the sysop does with all that computing power once it’s in charge. Sentience is one of the basic goods, but having whatever sentient creatures exist live excellent lives is another one. If the sysop uses 60% of the computing power to run itself and 40% to run sentient creatures, it could still be a net win if the sysop spends most of its time finding new ways to make those other creature’s lives as wonderful as possible.
Look at it another way, the organic matter currently being used to make your clothes, food, home, etc could probably also be used to make more humans out of. But it’s probably better to use it to improve your life then to create a bunch of cold, naked, hungry people.
Sentience is one of the basic goods. If the sysop is non-sentient, then whatever computronium is used in the sysop is, WRT sentience, wasted.
If we suppose that intelligences have a power-law distribution, and the sysop is the one at the top, we’ll find that it uses up something around 20% to 50% of the accessible universe’s computronium.
That would be a natural (as in “expected in nature”) distribution. But since the sysop needs to stay in charge, it will probably destroy any other AIs who reach the “second tier” of intelligence. So it will more likely have something like 70% − 90% of the universe’s computronium.
Also, in this post-human world, there aren’t large penalties for individuality. That is: In today’s world, you can’t add up 3 chimpanzee brains and get human-level intelligence. In the world of AIs, you probably can. This means that, to stay on top, the sysop will always need to reserve a majority of the universe’s computronium for itself. Otherwise, the rest of the universe can gang up on it.
So creating a non-sentient sysop means cutting the amount of sentient life you can support by at least half.
Not necessarily. It depends on what the sysop does with all that computing power once it’s in charge. Sentience is one of the basic goods, but having whatever sentient creatures exist live excellent lives is another one. If the sysop uses 60% of the computing power to run itself and 40% to run sentient creatures, it could still be a net win if the sysop spends most of its time finding new ways to make those other creature’s lives as wonderful as possible.
Look at it another way, the organic matter currently being used to make your clothes, food, home, etc could probably also be used to make more humans out of. But it’s probably better to use it to improve your life then to create a bunch of cold, naked, hungry people.