I feel like after reading this I have a much better insight into how Eliezer thinks than I did before, even having read most of his published work.
I think his model of other people is off though.
Specifically, he uses ideas of comparative status to explain other people not challenging conventional wisdom, or trying new things a lot. Which feels like it could be a fully general argument for any observed behaviour (e.g. it could equally well explain a habit of disproportionately challenging experts, as being in conflict with them puts you at their level and gains status).
I think its more accurate to say that people are reluctant to try new things because of loss aversion. Let’s say people are investing a certain amount of time and energy in a project, like writing HPMOR, one option has a 1⁄100 chance of a million-dollar (or utils or whatever) reward, the other is a certainty of a thousand dollars. Most people would choose the second, even if that’s not what the naive math would say. Similarly, if they thought that a project as big as writing a novel had a 9⁄10 chance of being a flop, then they wouldn’t even try it. (I definitely notice this habit in myself, rejections and failures weigh on me mentally more than successes. So, my behaviour often defaults to avoiding them.) The moderate position is to start with less investment heavy/risky projects first to get a better idea of likely success levels and build confidence.
I feel like after reading this I have a much better insight into how Eliezer thinks than I did before, even having read most of his published work.
I think his model of other people is off though.
Specifically, he uses ideas of comparative status to explain other people not challenging conventional wisdom, or trying new things a lot. Which feels like it could be a fully general argument for any observed behaviour (e.g. it could equally well explain a habit of disproportionately challenging experts, as being in conflict with them puts you at their level and gains status).
I think its more accurate to say that people are reluctant to try new things because of loss aversion. Let’s say people are investing a certain amount of time and energy in a project, like writing HPMOR, one option has a 1⁄100 chance of a million-dollar (or utils or whatever) reward, the other is a certainty of a thousand dollars. Most people would choose the second, even if that’s not what the naive math would say. Similarly, if they thought that a project as big as writing a novel had a 9⁄10 chance of being a flop, then they wouldn’t even try it. (I definitely notice this habit in myself, rejections and failures weigh on me mentally more than successes. So, my behaviour often defaults to avoiding them.) The moderate position is to start with less investment heavy/risky projects first to get a better idea of likely success levels and build confidence.