I agree that a decently fun world would still be pretty good, and if that maximizes expected value then that’s what we should choose, given a choice.
Of course, the “if/then” part of that sentence is important. In other words to go from there to “we should therefore choose a decently but not wildly overwhelmingly fun world, given a choice” is unjustified without additional data. Opportunity costs are still costs.
I agree that the world we live in is not unacceptably bad for everyone in it.
I agree that not everyone currently alive would be better off dead.
I agree that an AGI (as that term is used here) could potentially create a non-terrible circumstance for us to inhabit. (I’m not really sure what “relative to our preferences” means there. What else might “non-terrible” be relative to?)
I’m not at all sure why the two questions you list are “the” questions. I agree that the first one is worth answering. I’m not sure the second one is, though if we answer it along the way to doing something else that’s OK with me.
I agree that a decently fun world would still be pretty good, and if that maximizes expected value then that’s what we should choose, given a choice.
Of course, the “if/then” part of that sentence is important. In other words to go from there to “we should therefore choose a decently but not wildly overwhelmingly fun world, given a choice” is unjustified without additional data. Opportunity costs are still costs.
I agree that the world we live in is not unacceptably bad for everyone in it. I agree that not everyone currently alive would be better off dead.
I agree that an AGI (as that term is used here) could potentially create a non-terrible circumstance for us to inhabit. (I’m not really sure what “relative to our preferences” means there. What else might “non-terrible” be relative to?)
I’m not at all sure why the two questions you list are “the” questions. I agree that the first one is worth answering. I’m not sure the second one is, though if we answer it along the way to doing something else that’s OK with me.