Eliezer points out that it is impossible to have false beliefs only when they’re beneficial to you.
In theory it’s possible: look at what beliefs cause success, and adopt them. (Assuming you’re really good at determining causality between beliefs and success, really good at avoiding checking whether these beliefs are true, and really good at installing beliefs in your mind—but let’s assume the least convenient possible world for the sake of argument, where you are indeed really good at those things.)
Now whether our world looks like that or not is another question...
That’s just what Eliezer is pointing at with “Choosing to be Biased”. In practice, it’s impossible to determine causality between beliefs and “success”. There are lots of successful people with all kinds of utterly insane beliefs. Were their insane beliefs the cause of their success? Or were these people so talented that they were able to achieve success despite those beliefs?
In contrast, determining whether a belief results in good predictions about the world is quite easy, and gets you many of the benefits without any of the potentially catastrophic costs.
In theory it’s possible: look at what beliefs cause success, and adopt them. (Assuming you’re really good at determining causality between beliefs and success, really good at avoiding checking whether these beliefs are true, and really good at installing beliefs in your mind—but let’s assume the least convenient possible world for the sake of argument, where you are indeed really good at those things.)
Now whether our world looks like that or not is another question...
That’s just what Eliezer is pointing at with “Choosing to be Biased”. In practice, it’s impossible to determine causality between beliefs and “success”. There are lots of successful people with all kinds of utterly insane beliefs. Were their insane beliefs the cause of their success? Or were these people so talented that they were able to achieve success despite those beliefs?
In contrast, determining whether a belief results in good predictions about the world is quite easy, and gets you many of the benefits without any of the potentially catastrophic costs.
This makes me wonder. Does it mean we can’t determine if true beliefs cause success either? Or is there some important asymmetry?