Indeed, a utilitarian FAI that values Fun presumably immediately gets busy constructing a species more capable of Fun than humans, with the intention of populating the universe with them instead of us.
Oh.
I would hope that the FAI would instead turn us into the species most capable of fun. But considering the remaining time of the universe and all the fun the new species will have there, the difference between (a) transforming us or (b) killing us and creating the other species de novo, is negligible. The FAI would probably choose the faster solution, because it would allow more total fun-time for the superhappies. If there are more possible superhappy designs, equivalent in their fun-capacity, the FAI would chose the one that cares about us the least, to reduce their possible regret of our extinction. Probably something very unsimilar to us (as much as the definition of “fun” allows). They would care about us less than we care about the dinosaurs.
Faster would presumably be an issue, yes. Minimizing expected energy input per unit Fun output would presumably also be an issue.
Of course, all of this presumes that the FAI’s definition of Fun doesn’t definitionally restrict the experience of Fun to 21st-century humans (either as a species, or as a culture, or as individuals).
Unrelatedly, I’m not sure I agree about regret. I can imagine definitions of Fun such that maximizing Fun requires the capacity for regret, for example.
Oh.
I would hope that the FAI would instead turn us into the species most capable of fun. But considering the remaining time of the universe and all the fun the new species will have there, the difference between (a) transforming us or (b) killing us and creating the other species de novo, is negligible. The FAI would probably choose the faster solution, because it would allow more total fun-time for the superhappies. If there are more possible superhappy designs, equivalent in their fun-capacity, the FAI would chose the one that cares about us the least, to reduce their possible regret of our extinction. Probably something very unsimilar to us (as much as the definition of “fun” allows). They would care about us less than we care about the dinosaurs.
Faster would presumably be an issue, yes. Minimizing expected energy input per unit Fun output would presumably also be an issue.
Of course, all of this presumes that the FAI’s definition of Fun doesn’t definitionally restrict the experience of Fun to 21st-century humans (either as a species, or as a culture, or as individuals).
Unrelatedly, I’m not sure I agree about regret. I can imagine definitions of Fun such that maximizing Fun requires the capacity for regret, for example.