If you can’t think of anything medium-probable, using that as an excuse to do nothing is unacceptable.
That’s my gut feeling as well, but can we give a theoretical basis for that conclusion, which might also potentially be used to convince people who can’t think of anything medium-probable to “do something”?
My current thoughts are
I assign some non-zero credence to having an unbounded utility function.
Bostrom and Toby’s moral parliament idea seems to be the best that we have about how to handle moral uncertainty.
If Pascal’s wager argument works, and to the extent that I have a faction representing unbounded utility in my moral parliament, I ought to spend a fraction of my resources on Pascal’s wager type “opportunities”
If Pascal’s wager argument works, I should pick the best wager to bet on, which intuitively could well be “push for a positive Singularity”
But it’s not clear that Pascal’s wager argument works or what could be the justification for thinking that “push for a positive Singularity” is the best wager. We also don’t have any theory to handle this kind of philosophical uncertainty.
Given all this, I still have to choose between “do nothing”, “push for positive Singularity”, or “investigate Pascal’s wager”. Is there any way, in this decision problem, to improve upon going with my gut?
Anyway, I understand that you probably have reasons not to engage too deeply with this line of thought, so I’m mostly explaining where I’m currently at, as well as hoping that someone else can offer some ideas.
That’s my gut feeling as well, but can we give a theoretical basis for that conclusion, which might also potentially be used to convince people who can’t think of anything medium-probable to “do something”?
My current thoughts are
I assign some non-zero credence to having an unbounded utility function.
Bostrom and Toby’s moral parliament idea seems to be the best that we have about how to handle moral uncertainty.
If Pascal’s wager argument works, and to the extent that I have a faction representing unbounded utility in my moral parliament, I ought to spend a fraction of my resources on Pascal’s wager type “opportunities”
If Pascal’s wager argument works, I should pick the best wager to bet on, which intuitively could well be “push for a positive Singularity”
But it’s not clear that Pascal’s wager argument works or what could be the justification for thinking that “push for a positive Singularity” is the best wager. We also don’t have any theory to handle this kind of philosophical uncertainty.
Given all this, I still have to choose between “do nothing”, “push for positive Singularity”, or “investigate Pascal’s wager”. Is there any way, in this decision problem, to improve upon going with my gut?
Anyway, I understand that you probably have reasons not to engage too deeply with this line of thought, so I’m mostly explaining where I’m currently at, as well as hoping that someone else can offer some ideas.