We do in fact want to save worlds we can’t begin to fathom from dangers we can’t begin to fathom even if it makes us depressed or dead… but if you don’t get any satisfaction from saving the world, you might have a problem with selfishness.
...but if you don’t get any satisfaction from saving the world, you might have a problem with selfishness.
That’s not what I meant. What I meant is the general problem you run into when you take this stuff to its extreme. You end up saving hypothetical beings with a very low probability. That means that you might very well save no being at all, if your model was bogus. I am aware that the number of beings saved often outweighs the low probability...but I am not particular confident in this line of reasoning, i.e. in the meta-level of thinking about how to maximize good deeds. That leads to all kind of crazy seeming stuff.
If it does, something almost definitely went wrong. Biases crept in somewhere between the risk assessment, the outside view correction process, the policy-proposing process, the policy-analyzing process, the policy outside view correction process, the ethical injunction check, and the “(anonymously) ask a few smart people whether some part of this is crazy” step. I’m not just adding unnatural steps; each of those should be separate, and each of those is a place where error can throw everything off. Overconfidence plus conjunction fallacy equals crazy seeming stuff. And this coming from the guy who is all about taking ideas seriously.
We do in fact want to save worlds we can’t begin to fathom from dangers we can’t begin to fathom even if it makes us depressed or dead… but if you don’t get any satisfaction from saving the world, you might have a problem with selfishness.
That’s not what I meant. What I meant is the general problem you run into when you take this stuff to its extreme. You end up saving hypothetical beings with a very low probability. That means that you might very well save no being at all, if your model was bogus. I am aware that the number of beings saved often outweighs the low probability...but I am not particular confident in this line of reasoning, i.e. in the meta-level of thinking about how to maximize good deeds. That leads to all kind of crazy seeming stuff.
If it does, something almost definitely went wrong. Biases crept in somewhere between the risk assessment, the outside view correction process, the policy-proposing process, the policy-analyzing process, the policy outside view correction process, the ethical injunction check, and the “(anonymously) ask a few smart people whether some part of this is crazy” step. I’m not just adding unnatural steps; each of those should be separate, and each of those is a place where error can throw everything off. Overconfidence plus conjunction fallacy equals crazy seeming stuff. And this coming from the guy who is all about taking ideas seriously.