A partial way around this is to not double-count the basic evidence—if a person’s belief is based on some evidence, count the evidence, not the belief.
If person A agrees with you but has exactly the same evidence as you, this should only slightly change your beliefs, reducing your estimate chance of making a mistake or being insane—it doesn’t actually say anything new about the evidence. Remember, ideal logicians count each piece of evidence only once, even if it comes from someone else’s mouth. If person B disagrees with you, is rational, and has a different set of evidence, you should change your opinion to reflect the new evidence.
Also, people who disagree with you about things you don’t have much evidence about have a reasonably good chance of being better informed than you on the subject, or at least having different evidence about the subject.
Moreover, the correct answer based on all the evidence may not even look anything like the “average” of the beliefs. In the trivial case, imagine two people see a coin and both observe different flipping events where a handful of heads come up. Both update in the direction of a biased coin, but aren’t certain. After sharing data each should be more certain than either was alone that the coin is biased towards heads.
In theory, seeking out people whose beliefs are different from your own, to ask them about their knowledge and experiences should make you better informed than doing the same with people with whom you already agree with about most things. They don’t even have to be “rationalists” for it to benefit you they only need to be honest about their experiences...
On the other hand, they might need to be rationalists to reciprocally benefit from talking with you.
A partial way around this is to not double-count the basic evidence—if a person’s belief is based on some evidence, count the evidence, not the belief.
If person A agrees with you but has exactly the same evidence as you, this should only slightly change your beliefs, reducing your estimate chance of making a mistake or being insane—it doesn’t actually say anything new about the evidence. Remember, ideal logicians count each piece of evidence only once, even if it comes from someone else’s mouth. If person B disagrees with you, is rational, and has a different set of evidence, you should change your opinion to reflect the new evidence.
Also, people who disagree with you about things you don’t have much evidence about have a reasonably good chance of being better informed than you on the subject, or at least having different evidence about the subject.
Moreover, the correct answer based on all the evidence may not even look anything like the “average” of the beliefs. In the trivial case, imagine two people see a coin and both observe different flipping events where a handful of heads come up. Both update in the direction of a biased coin, but aren’t certain. After sharing data each should be more certain than either was alone that the coin is biased towards heads.
In theory, seeking out people whose beliefs are different from your own, to ask them about their knowledge and experiences should make you better informed than doing the same with people with whom you already agree with about most things. They don’t even have to be “rationalists” for it to benefit you they only need to be honest about their experiences...
On the other hand, they might need to be rationalists to reciprocally benefit from talking with you.