This section is probably my favorite thing you (Tsvi) have written, and motivated me to read through all your alignment related posts on your blog.
Before I read that passage, I was confident that deconfusion research was the highest value thing I could be doing (and getting better at), but I did not have a succinct way of communicating the fact that me seeming confused about a certain concept is not a sign that I have worse understanding about the problem involved compared to someone who doesn’t seem confused.
There’s a misconception where most people pattern match confidence in one’s understanding of a concept / domain with better understanding of the domain, while vagueness in description of a concept as someone not quite understanding the domain. I notice hints of these even in rationalist friends I have, the ones who have read The Sequences and have a strong aversion to stuff that, in their head, pattern matches to making basic rationality mistakes. Reading this passage helped me have a handle on why I felt that my epistemic state was still better than that of others who seemed more confident in their claims.
This section is probably my favorite thing you (Tsvi) have written, and motivated me to read through all your alignment related posts on your blog.
Before I read that passage, I was confident that deconfusion research was the highest value thing I could be doing (and getting better at), but I did not have a succinct way of communicating the fact that me seeming confused about a certain concept is not a sign that I have worse understanding about the problem involved compared to someone who doesn’t seem confused.
There’s a misconception where most people pattern match confidence in one’s understanding of a concept / domain with better understanding of the domain, while vagueness in description of a concept as someone not quite understanding the domain. I notice hints of these even in rationalist friends I have, the ones who have read The Sequences and have a strong aversion to stuff that, in their head, pattern matches to making basic rationality mistakes. Reading this passage helped me have a handle on why I felt that my epistemic state was still better than that of others who seemed more confident in their claims.
Also, I feel like this somewhat relates to Eliezer’s aversion to bio-anchors and concrete ‘base rates’, but I don’t yet have a good way of clarifying it in my head.