Eliezer emailed me to ask me about it (per Carl’s request, above); I emailed him back with the email below, which Eliezer requested I paste into the LW thread. Pasting:
In the majority of cases, people do not straightforwardly say “X is false, but I need to believe X anyhow”. More often they wiggle, and polite conversation drops the subject.
You have made statements that I and at least some others interpreted as perhaps indicating such wiggles (i.e., as perhaps indicating a desire to hold onto false impressions by diverting conscious attention or discussion from a particular subject). You have never, to my knowledge, uttered a straight-forward endorsement of holding false beliefs. The wiggle-suggesting statements were not super-clear, and were not beyond the realm of possible misinterpretation.
Re: statements that seemed to me and some others to indicate possible wiggles: You have mentioned multiple times that, well, I forget, but something like, it’d be hard to do top focused research on FAI-like problems while estimating a 1% chance of success. You’ve also avoided giving probability estimates in a lot of contexts, and have sometimes discouraged conversations in which others did so. You seemed perhaps a bit upset or defensive at multiple points during the probability estimates conversation with Zvi in NYC (enough so that a couple people commented on it with surprise to me afterward (not Carl or Paul; I forget who; newcomers to our conversations)), but, to your credit, you commented on these difficulties and proceeded to use debiasing techniques (e.g., I think you might’ve mentioned leaving a line of retreat, and might’ve given yourself a minute to do so).
If you would like polite conversation not to drop the subject on future occasions on which your impressions look (to me and other non-telepathic observers) like they might possibly be wiggly, give me a straight-forward request and I can change my settings here. I have in fact been a bit afraid of disturbing your motivation, and also a bit afraid of reducing your good will toward me.
Michael Vassar might be another interesting one to probe, if you’re collecting opinions. Though note that Vassar, Carl, and I have all discussed this at least a bit, and so are not independent datapoints.
From my internal perspective, the truth-as-I-experience-it is that I’m annoyed when people raise the topic because it’s all wasted motion, the question sets up a trap that forces you into appearing arrogant, and I honestly think that “Screw all this, I’m just going to go ahead and do it and you can debate afterward what the probabilities were” is a perfectly reasonable response.
From the perspective of folks choosing between supporting multiple lines of AI risk reduction effort, of which MIRI is only one, such probability estimates are not wasted effort.
Though your point about appearing arrogant is well taken. It’s unfortunate that it isn’t socially okay to publicly estimate a high probability of success, or to publicly claim one’s own exceptionalism, when ones impressions point that way. It places a barrier toward honest conversation here.
From my internal perspective, the truth-as-I-experience-it is that I’m annoyed when people raise the topic [of MIRI’s success-odds] because [good reason].
I suspect this annoyance is easily misinterpreted, independent of its actual cause. Most humans respond with annoyance when their plans are criticized. Also, in situations where A has power over B, and where B then shares concerns or criticisms about A’s plans, and where A responds with annoyance or with avoidance of such conversation… B is apt to respond (as I did) by being a bit hesitant to bring the topic up, and by also wondering if A is being defensive.
I’m not saying I was correct here. I’m also not sure what the fix is. But it might be worth setting a 1-minute timer and brainstorming or something.
Eliezer emailed me to ask me about it (per Carl’s request, above); I emailed him back with the email below, which Eliezer requested I paste into the LW thread. Pasting:
From my internal perspective, the truth-as-I-experience-it is that I’m annoyed when people raise the topic because it’s all wasted motion, the question sets up a trap that forces you into appearing arrogant, and I honestly think that “Screw all this, I’m just going to go ahead and do it and you can debate afterward what the probabilities were” is a perfectly reasonable response.
From the perspective of folks choosing between supporting multiple lines of AI risk reduction effort, of which MIRI is only one, such probability estimates are not wasted effort.
Though your point about appearing arrogant is well taken. It’s unfortunate that it isn’t socially okay to publicly estimate a high probability of success, or to publicly claim one’s own exceptionalism, when ones impressions point that way. It places a barrier toward honest conversation here.
I suspect this annoyance is easily misinterpreted, independent of its actual cause. Most humans respond with annoyance when their plans are criticized. Also, in situations where A has power over B, and where B then shares concerns or criticisms about A’s plans, and where A responds with annoyance or with avoidance of such conversation… B is apt to respond (as I did) by being a bit hesitant to bring the topic up, and by also wondering if A is being defensive.
I’m not saying I was correct here. I’m also not sure what the fix is. But it might be worth setting a 1-minute timer and brainstorming or something.