And all of this is asserted as, essentially, obvious and undeniable, extreme confidence is displayed, all the arguments offered against this are invalid and dumb, and those that disagree are at best deeply confused and constantly told they did not understand or fairly represent what was said.
This feels unnecessarily snarky, but is also pretty much exactly the experience a lot of people have trying to engage with Yudkowsky et al. It feels weird to bring up “they’re very confident and say that their critics just don’t get it” as a put-down here.
It seems doubly bad because it really seems like a lot of the more pessimist crowd just genuinely aren’t actually trying to engage with these ideas at all. Nate wrote one skimmed post which badly misread the piece, and Yudkowsky AFAICT has at most engaged via a couple tweets (again which don’t seem to engage with the points). This is concurrent with them both engaging much more heavily with weaker objections to which they already have easy answers.
I genuinely don’t understand why a group which is highly truth-seeking and dispassionately interested in the validity of their very consequential arguments feels so little reason to engage with counter-arguments to their core claims which have been well-received.
I tried one reply to one of Pope’s posts
From your post, you seem to have misunderstood Quintin’s arguments in a way he explains pretty clearly, and then there’s not really much follow-up. You don’t seem to have demonstrated you can pass an ITT after this, and I think if it were Yudkowsky in Pope’s position and someone effectively wrote him off as hopeless after one failed attempt to understand eachother you would probably not be as forgiving.
From my perspective here’s what happened: I spent hours trying to parse his arguments. I then wrote an effort post, responding to something that seemed very wrong to me, that took me many hours, that was longer than the OP, and attempted to explore the questions and my model in detail.
He wrote a detailed reply, which I thanked him for, ignoring the tone issues in question here and focusing on thee details and disagreements. I spent hours processing it and replied in detail to each of his explanations in the reply, including asking many detailed questions, identifying potential cruxes, making it clear where I thought he was right about my mistakes, and so on. I read all the comments carefully, by everyone.
This was an extraordinary, for me, commitment of time, by this point, while the whole thing was stressful. He left it at that. Which is fine, but I don’t know how else I was supposed to ‘follow up’ at that point? I don’t know what else someone seeking to understand is supposed to do.
I agree Nate’s post was a mistake, and said so in OP here—either take the time to engage or don’t engage. That was bad. But in general no, I do not think that the thing I am observing from Pope/Belrose is typical of LW/AF/rationalist/MIRI/etc behaviors to anything like the same degree that they consistently do it.
Nor do I get the sense that they are open to argument. Looking over Pope’s reply to me, I basically don’t see him changing his mind about anything, agreeing a good point was made, addressing my arguments or thoughts on their merits rather than correcting my interpretation of his arguments, asking me questions, suggesting cruxes and so on. Where he notes disagreement he says he’s baffled anyone could think such a thing and doesn’t seem curious why I might think it.
If people want to make a higher bid for me to engage more after that, I am open to hearing it. Otherwise, I don’t see how to usefully do so in reasonable time in a way that would have value.
Sorry you found it so stressful! I’m not objecting to you deciding it’s not worth your time to engage, what I’m getting at is a perceived double standard in when this kind of criticism is applied. You say
I do not think that the thing I am observing from Pope/Belrose is typical of LW/AF/rationalist/MIRI/etc behaviors to anything like the same degree that they consistently do it
But this seems wrong to me. The best analogue of your post from Quintin’s perspective was his own post laying out disagreements with Eliezer. Eliezer’s response to this was to say it was too long for him to bother reading, which imo is far worse. AFAICT his response to you in your post is higher-effort than the responses from MIRI people to his arguments all put together. Plausibly we have different clusters in our head of who we’re comparing him too though—I agree a wider set of LW people are much more engaging, I’m specifically comparing to e.g Nate and Eliezer as that feels to me a fairer comparison
To go into the specific behaviours you mention
I basically don’t see him changing his mind about anything, agreeing a good point was made
I don’t think this makes sense—if from his perspective you didn’t make good points or change his mind then what was he supposed to do? If you still think you did and he’s not appreciating them then that’s fair but is more reifying the initial disagreement. I also don’t see this behaviour from Eliezer or Nate?
addressing my arguments or thoughts on their merits rather than correcting my interpretation of his arguments, asking me questions, suggesting cruxes and so on.
I again don’t see Eliezer doing any of this either in responses to critical posts?
Where he notes disagreement he says he’s baffled anyone could think such a thing and doesn’t seem curious why I might think it
Again seems to be a feature of many MIRI-cluster responses. Stating that certain things feel obvious from the inside and that you don’t get why it’s so hard for other people to grok them is a common refrain.
I genuinely don’t understand why a group which is highly truth-seeking and dispassionately interested in the validity of their very consequential arguments feels so little reason to engage with counter-arguments to their core claims which have been well-received.
A bunch of the more pessimistic people have in practice spent a decent amount of time trying to argue with (e.g.) Paul Christiano and other people who are more optimistic. So, it’s not as though the total time spent engaging with counter-arguments is small.
Additionally, I think there are basically two different questions here:
Should people who are very pessimistic be interested in spending a bunch of time engaging with counterarguments? This could be either to argue to bystanders or get a better model of reality for themselves and/or the counterparty.
Who should people who are very pessimistic aim to engage with and what counterarguments should they discuss?
I’m pretty sympathetic to the take that pessimistic people should spend more time engaging (1), but not that sure that immediately engaging with specifically “AI optimists” is the best approach (2).
(FWIW, I think both AI optimists and Yudkowsky and Nate often make important errors wrt. various arguments at least when these arguments are made publicly in writing.)
and Yudkowsky AFAICT has at most engaged via a couple tweets (again which don’t seem to engage with the points).
As a point of data, Yudkowsky also responded in a very terse manner and basically didn’t respond to Pope’s post at all, so this is not a one-off event:
Yudkowsky AFAICT has at most engaged via a couple tweets (again which don’t seem to engage with the points).
If you mean literally two, it’s more, although I won’t take the time to dig up the tweets. I remember seeing them discuss at non-trivial length at least once on twitter. (If “a couple” encompassed that… Well once someone asked me “a couple of spaghetti” and when I gave him 2 spaghetti he got quite upset. Uhm. Don’t get upset at me, please?)
I’ve thought a bit about this because I too on first sight perceived a lack of serious engagement. I’ve not yet come to a confident conclusion; on reflection I’m not so sure anymore there was an unfair lack of engagement.
First I tried to understand Pope’s & co arguments at the object level. Within the allotted time, I failed. I expected to fare better, so I think there’s some mixture of (Pope’s framework not being simplifiable) & (Pope’s current communication situation low), where the comparatives refer to the state of Yudkowsky’s & co framework when I first encountered it.
So I turned to proxies; in cases where I thought I understood the exchange, what could I say about it? Did it seem fair?
From this I got the impression that sometimes Pope makes blunders at understanding simple things Yudkowsky means (not cruxes or anything really important, just trivial misunderstandings), which throw a shadow over his reading comprehension, such that then one is less inclined to spend the time to take him seriously when he makes complicated arguments that are not clear at once.
On the other hand, Yudkowsky seems to not take the time to understand when Pope’s prose is a bit approximative or not totally rigorous, which is difficult to avoid when compressing technical arguments.
So my current conceit is: a mixture of (Pope is not good at communicating) & (does not invest in communication). This does not bear significatively on whether he’s right, but it’s a major time investment to understand him, so inevitably someone with many options on who to talk to is gonna deprioritize him.
To look at a more specific point, Vaniver replied at length to Quintin’s post on Eliezer’s podcast, and Eliezer said those answers were already “pretty decent”, so although he did not take the time to answer personally, he bothered to check that someone was replying more thoroughly.
P.S. to try to be actionable: I think Pope’s viewpoint would greatly benefit from having someone who understands it well, but is good and dedicated at communication. Although they are faring quite well on fame, so maybe they don’t need, after all, anything more?
P.P.S. they now have a website, optimists.ai, so indeed they do think they should ramp up communication efforts, instead of resting on their current level of fame.
This feels unnecessarily snarky, but is also pretty much exactly the experience a lot of people have trying to engage with Yudkowsky et al. It feels weird to bring up “they’re very confident and say that their critics just don’t get it” as a put-down here.
It seems doubly bad because it really seems like a lot of the more pessimist crowd just genuinely aren’t actually trying to engage with these ideas at all. Nate wrote one skimmed post which badly misread the piece, and Yudkowsky AFAICT has at most engaged via a couple tweets (again which don’t seem to engage with the points). This is concurrent with them both engaging much more heavily with weaker objections to which they already have easy answers.
I genuinely don’t understand why a group which is highly truth-seeking and dispassionately interested in the validity of their very consequential arguments feels so little reason to engage with counter-arguments to their core claims which have been well-received.
From your post, you seem to have misunderstood Quintin’s arguments in a way he explains pretty clearly, and then there’s not really much follow-up. You don’t seem to have demonstrated you can pass an ITT after this, and I think if it were Yudkowsky in Pope’s position and someone effectively wrote him off as hopeless after one failed attempt to understand eachother you would probably not be as forgiving.
From my perspective here’s what happened: I spent hours trying to parse his arguments. I then wrote an effort post, responding to something that seemed very wrong to me, that took me many hours, that was longer than the OP, and attempted to explore the questions and my model in detail.
He wrote a detailed reply, which I thanked him for, ignoring the tone issues in question here and focusing on thee details and disagreements. I spent hours processing it and replied in detail to each of his explanations in the reply, including asking many detailed questions, identifying potential cruxes, making it clear where I thought he was right about my mistakes, and so on. I read all the comments carefully, by everyone.
This was an extraordinary, for me, commitment of time, by this point, while the whole thing was stressful. He left it at that. Which is fine, but I don’t know how else I was supposed to ‘follow up’ at that point? I don’t know what else someone seeking to understand is supposed to do.
I agree Nate’s post was a mistake, and said so in OP here—either take the time to engage or don’t engage. That was bad. But in general no, I do not think that the thing I am observing from Pope/Belrose is typical of LW/AF/rationalist/MIRI/etc behaviors to anything like the same degree that they consistently do it.
Nor do I get the sense that they are open to argument. Looking over Pope’s reply to me, I basically don’t see him changing his mind about anything, agreeing a good point was made, addressing my arguments or thoughts on their merits rather than correcting my interpretation of his arguments, asking me questions, suggesting cruxes and so on. Where he notes disagreement he says he’s baffled anyone could think such a thing and doesn’t seem curious why I might think it.
If people want to make a higher bid for me to engage more after that, I am open to hearing it. Otherwise, I don’t see how to usefully do so in reasonable time in a way that would have value.
Sorry you found it so stressful! I’m not objecting to you deciding it’s not worth your time to engage, what I’m getting at is a perceived double standard in when this kind of criticism is applied. You say
But this seems wrong to me. The best analogue of your post from Quintin’s perspective was his own post laying out disagreements with Eliezer. Eliezer’s response to this was to say it was too long for him to bother reading, which imo is far worse. AFAICT his response to you in your post is higher-effort than the responses from MIRI people to his arguments all put together. Plausibly we have different clusters in our head of who we’re comparing him too though—I agree a wider set of LW people are much more engaging, I’m specifically comparing to e.g Nate and Eliezer as that feels to me a fairer comparison
To go into the specific behaviours you mention
I don’t think this makes sense—if from his perspective you didn’t make good points or change his mind then what was he supposed to do? If you still think you did and he’s not appreciating them then that’s fair but is more reifying the initial disagreement. I also don’t see this behaviour from Eliezer or Nate?
I again don’t see Eliezer doing any of this either in responses to critical posts?
Again seems to be a feature of many MIRI-cluster responses. Stating that certain things feel obvious from the inside and that you don’t get why it’s so hard for other people to grok them is a common refrain.
A bunch of the more pessimistic people have in practice spent a decent amount of time trying to argue with (e.g.) Paul Christiano and other people who are more optimistic. So, it’s not as though the total time spent engaging with counter-arguments is small.
Additionally, I think there are basically two different questions here:
Should people who are very pessimistic be interested in spending a bunch of time engaging with counterarguments? This could be either to argue to bystanders or get a better model of reality for themselves and/or the counterparty.
Who should people who are very pessimistic aim to engage with and what counterarguments should they discuss?
I’m pretty sympathetic to the take that pessimistic people should spend more time engaging (1), but not that sure that immediately engaging with specifically “AI optimists” is the best approach (2).
(FWIW, I think both AI optimists and Yudkowsky and Nate often make important errors wrt. various arguments at least when these arguments are made publicly in writing.)
ETA: there is relevant context in this post from Nate: Hashing out long-standing disagreements seems low-value to me
As a point of data, Yudkowsky also responded in a very terse manner and basically didn’t respond to Pope’s post at all, so this is not a one-off event:
https://www.lesswrong.com/posts/wAczufCpMdaamF9fy/my-objections-to-we-re-all-gonna-die-with-eliezer-yudkowsky#YYR4hEFRmA7cb5csy
If you mean literally two, it’s more, although I won’t take the time to dig up the tweets. I remember seeing them discuss at non-trivial length at least once on twitter. (If “a couple” encompassed that… Well once someone asked me “a couple of spaghetti” and when I gave him 2 spaghetti he got quite upset. Uhm. Don’t get upset at me, please?)
I’ve thought a bit about this because I too on first sight perceived a lack of serious engagement. I’ve not yet come to a confident conclusion; on reflection I’m not so sure anymore there was an unfair lack of engagement.
First I tried to understand Pope’s & co arguments at the object level. Within the allotted time, I failed. I expected to fare better, so I think there’s some mixture of (Pope’s framework not being simplifiable) & (Pope’s current communication situation low), where the comparatives refer to the state of Yudkowsky’s & co framework when I first encountered it.
So I turned to proxies; in cases where I thought I understood the exchange, what could I say about it? Did it seem fair?
From this I got the impression that sometimes Pope makes blunders at understanding simple things Yudkowsky means (not cruxes or anything really important, just trivial misunderstandings), which throw a shadow over his reading comprehension, such that then one is less inclined to spend the time to take him seriously when he makes complicated arguments that are not clear at once.
On the other hand, Yudkowsky seems to not take the time to understand when Pope’s prose is a bit approximative or not totally rigorous, which is difficult to avoid when compressing technical arguments.
So my current conceit is: a mixture of (Pope is not good at communicating) & (does not invest in communication). This does not bear significatively on whether he’s right, but it’s a major time investment to understand him, so inevitably someone with many options on who to talk to is gonna deprioritize him.
To look at a more specific point, Vaniver replied at length to Quintin’s post on Eliezer’s podcast, and Eliezer said those answers were already “pretty decent”, so although he did not take the time to answer personally, he bothered to check that someone was replying more thoroughly.
P.S. to try to be actionable: I think Pope’s viewpoint would greatly benefit from having someone who understands it well, but is good and dedicated at communication. Although they are faring quite well on fame, so maybe they don’t need, after all, anything more?
P.P.S. they now have a website, optimists.ai, so indeed they do think they should ramp up communication efforts, instead of resting on their current level of fame.