Obviously Eliezer thinks that the people who agree with the arguments that convince him are intelligent. Valuing people who can show your cherished arguments to be wrong is very nearly a post-human trait—it is extraordinarily rare among humans, and even then unevenly manifested.
On the other hand, if we are truly dedicated to overcoming bias, then we should value such people even more highly than those whom we can convince to question or abandon their cherished (but wrong) arguments/beliefs.
The problem is figuring out who those people are.
But it’s very difficult. If someone can correctly argue me out of an incorrect position, then they must understand the question better than I do, which makes it difficult or impossible for me to judge their information. Maybe they just swindled me, and my initial naive interpretation is really correct, while their argument has a serious flaw that someone more schooled than I would recognize?
So I’m forced to judge heuristically by signs of who can be trusted.
I tentatively believe that a strong sign of a person who can help me revise my beliefs is a person who is willing to revise their beliefs in the face of argument.
Eliezer’s descriptions of his intellectual history and past mistakes are very convincing positive signals to me. The occasional mockery and disdain for those who disagree is a bit of a negative signal.
But this comment here is not a negative signal at all, for me. Why? Because even if Eliezer was wrong, the other party’s willingness to reexamine is a strong signal of intelligence. Confirmation bias is so strong, that the willingness to act against it is of great value, even if this sometimes leads to greater error. A limited, faulty error correction mechanism (with some positive average value) is dramatically better than no error correction mechanism in the long run.
So yes, if I can (honestly) convince a person to question something that they previously deeply held, that is a sign of intelligence on their part. Agreeing with me is not the signal. Changing their mind is the signal.
It would be a troubling sign for me if there were no one who could convince me to change any of my deeply held beliefs.
Obviously Eliezer thinks that the people who agree with the arguments that convince him are intelligent. Valuing people who can show your cherished arguments to be wrong is very nearly a post-human trait—it is extraordinarily rare among humans, and even then unevenly manifested.
On the other hand, if we are truly dedicated to overcoming bias, then we should value such people even more highly than those whom we can convince to question or abandon their cherished (but wrong) arguments/beliefs.
The problem is figuring out who those people are.
But it’s very difficult. If someone can correctly argue me out of an incorrect position, then they must understand the question better than I do, which makes it difficult or impossible for me to judge their information. Maybe they just swindled me, and my initial naive interpretation is really correct, while their argument has a serious flaw that someone more schooled than I would recognize?
So I’m forced to judge heuristically by signs of who can be trusted.
I tentatively believe that a strong sign of a person who can help me revise my beliefs is a person who is willing to revise their beliefs in the face of argument.
Eliezer’s descriptions of his intellectual history and past mistakes are very convincing positive signals to me. The occasional mockery and disdain for those who disagree is a bit of a negative signal.
But this comment here is not a negative signal at all, for me. Why? Because even if Eliezer was wrong, the other party’s willingness to reexamine is a strong signal of intelligence. Confirmation bias is so strong, that the willingness to act against it is of great value, even if this sometimes leads to greater error. A limited, faulty error correction mechanism (with some positive average value) is dramatically better than no error correction mechanism in the long run.
So yes, if I can (honestly) convince a person to question something that they previously deeply held, that is a sign of intelligence on their part. Agreeing with me is not the signal. Changing their mind is the signal.
It would be a troubling sign for me if there were no one who could convince me to change any of my deeply held beliefs.