One important piece of information that Pat doesn’t seem to be registering is the part where Eliezer is already working on the project and feels that it’s going well. This added fact contains a lot of important information and Pat glosses over it.
If a physicist friend tells me they think they could probably solve [longstanding important physics problem] if they tried, I will be about as skeptical as if they claimed that they could do some other arbitrary difficult thing. If they tell me they have the solution to [longstanding important physics problem] clearly in mind, they’ve been working on developing the details and that work is going well, and they’re 10% confident that they will succeed, then I’ll probably update pretty far towards their own estimate.
I agree with this. There are a lot of little clues like this pointing in one direction or the other. I think a lot of people have a learned helplessness about noticing and combating biases (both in themselves, and in other people), and that this results in people under-updating on local evidence that someone is more/less likely to be dissembling or deceiving themselves.
I’m dealing with cognitive processes in my environment that are frequently adversarial and trying to “cheat” (if only unconsciously), but I nonetheless have to be able to update on evidence against rationalization just as readily as evidence for rationalization. (And in my case, I have to try to consciously correct for tendencies like “I’m more likely to be sensitive to evidence of cheating, and insensitive to evidence of non-cheating, when the person claiming to be able to outperform feels low-status to me, or when it feels like they’re doing something socially risky.”)
This added fact contains a lot of important information and Pat glosses over it.
Ah, but this fact isn’t third-party visible, which is one of the requirements Pat has before they’ll accept it as valid evidence. (Since, you know, anyone can say that they feel it’s going well, right? What makes you feel especially confident about your own judgment as to whether or not a given project is going well?)
One important piece of information that Pat doesn’t seem to be registering is the part where Eliezer is already working on the project and feels that it’s going well. This added fact contains a lot of important information and Pat glosses over it.
If a physicist friend tells me they think they could probably solve [longstanding important physics problem] if they tried, I will be about as skeptical as if they claimed that they could do some other arbitrary difficult thing. If they tell me they have the solution to [longstanding important physics problem] clearly in mind, they’ve been working on developing the details and that work is going well, and they’re 10% confident that they will succeed, then I’ll probably update pretty far towards their own estimate.
I agree with this. There are a lot of little clues like this pointing in one direction or the other. I think a lot of people have a learned helplessness about noticing and combating biases (both in themselves, and in other people), and that this results in people under-updating on local evidence that someone is more/less likely to be dissembling or deceiving themselves.
I’m dealing with cognitive processes in my environment that are frequently adversarial and trying to “cheat” (if only unconsciously), but I nonetheless have to be able to update on evidence against rationalization just as readily as evidence for rationalization. (And in my case, I have to try to consciously correct for tendencies like “I’m more likely to be sensitive to evidence of cheating, and insensitive to evidence of non-cheating, when the person claiming to be able to outperform feels low-status to me, or when it feels like they’re doing something socially risky.”)
Ah, but this fact isn’t third-party visible, which is one of the requirements Pat has before they’ll accept it as valid evidence. (Since, you know, anyone can say that they feel it’s going well, right? What makes you feel especially confident about your own judgment as to whether or not a given project is going well?)
It is, of course, third-party visible that Eliezer-2010 *says* it’s going well. Anyone can say that, but not everyone does.