I said: “The reason I got interested in UIVs to start with is that I didn’t have a good way to decide what counted as a good outcome.”
So I have realized that perhaps the best prophylactic against anthropic optimism bias is, in fact, to be genuinely unsure of what outcome you think is best. If you don’t have a predetermined idea of what to argue in favor of, then you don’t have a preferred outcome to argue in favor of. Admittedly this is not something that one can always do, but in the specific case of trying to work out what kind of goals to program an AI with, I think it is possible. Of course there are some outcomes that I don’t want, like a universe full of nothing but paperclips. But for the vast majority of “interesting” outcomes, I am still looking for a way to decide.
TGGP: “No, let me guess, you’ll be more productive if you’re happy Deja vu.”—thanks for the link! So, it seems that yet another person has had roughly the same idea as me...
I said: “The reason I got interested in UIVs to start with is that I didn’t have a good way to decide what counted as a good outcome.”
So I have realized that perhaps the best prophylactic against anthropic optimism bias is, in fact, to be genuinely unsure of what outcome you think is best. If you don’t have a predetermined idea of what to argue in favor of, then you don’t have a preferred outcome to argue in favor of. Admittedly this is not something that one can always do, but in the specific case of trying to work out what kind of goals to program an AI with, I think it is possible. Of course there are some outcomes that I don’t want, like a universe full of nothing but paperclips. But for the vast majority of “interesting” outcomes, I am still looking for a way to decide.
TGGP: “No, let me guess, you’ll be more productive if you’re happy Deja vu.”—thanks for the link! So, it seems that yet another person has had roughly the same idea as me...