Sorry, I think I can do better with a little more wordcount and effort.
I think that you think that the reason Dai is wrong to implicitly ask “Why not just not defer in this case?” is because you think that’s not relevantly on-topic for a post about how to mitigate harms from deference by an author who has established that he understands why deference is harmful, because you think that the implied question falsely presupposes that the post author is not aware of why deference is harmful.
(Whereas I think, and I think that Dai thinks, that the question is relevantly on-topic, because even if everyone in the conversation agrees on the broad outlines of why deference is harmful, they might disagree on the nitty-gritty details of exactly how harmful and exactly why, and litigating the nitty-gritty details of an example that was brought up in the post might help in evaluating the post’s thesis.)
Thank you. IDK if it’s closer, it’s still quite incorrect, but I greatly appreciate the effort.
[Edit: Just to state this: I think Dai was coming from good intentions, e.g. to think about important and interesting things. On reflection: I stand by something being unhelpful about the behavior pattern that I’m claiming I have engaged in, that you have engaged in, and that I claim Dai has engaged in; but also, I don’t think it’s some egregious transgression; I think I overreacted and I would like figure out how to handle discussion more gracefully and non-antagonistically; I value open thoughtful critical discussion and am grateful when people are interested in discussing things related to my writing. I also sorta meant this OP as being somewhat playful, but also I was annoyed, so it might have been more antagonizing than ideal.]
[Edit 2: I’m going to go through some of the points in more detail, because it seems like I haven’t successfully explained the pattern clearly enough yet, so I want to try to make things clearer to Zack. But, I don’t want this detailed wall of text to imply that the situation is some big deal; the behavior I’m discussing is not that bad even if you completely buy my perspective; I don’t mean to harp on it, I just want to answer Zack’s questions.]
Dai is wrong to implicitly ask “Why not just not defer in this case?”
I do not think Dai is wrong to ask that question. I think it’s a good and important question.
I don’t think the question was super inexplicit. In his first comment, the last sentence was “In other words, if you were going to spend your career on AI x-safety, of course you could have become an expert on these questions first.”. I followed up with a response to that.
I would say that the fact that his first comment quoted my statement about “Yudkowsky = best AGI X-derisk strategist”, and most of the words were arguing against that, confused the topic. I thought that literal statement was what he wanted to argue against, because he quoted it. Subsequently he did not seem to engage in that specific question, instead starting to develop a different question about different aspects of being a strategic thinker (which is also interesting, but even more off-topic, which is fine, but also confusing).
you think that the reason Dai is wrong to implicitly ask “Why not just not defer in this case?” is because you think that’s not relevantly on-topic
I do think it’s off-topic, but again, being off-topic is fine. I think it’s polite to acknowledge this, e.g. sometimes if I’m responding to something other than one of the main threads of a post I will (or think I ought to even if I don’t) say “Nitpick:” or “Off-topic, but:” or “I didn’t read the post carefully, just responding to [quote]:”. But being maybe mildly impolite in this way is not by itself the issue I have with that thread (though it is one element of the pattern I describe in the OP).
because you think that the implied question falsely presupposes that the post author is not aware of why deference is harmful
Let me venture a description of the events from Dai’s perspective. I will write Imaginary!Dai to emphasize that this is not Dai’s perspective, but my attempt to guess a plausible-to-me way his experience might have been. [I should maybe have similarly used Straw!Dai in the dialogue in the OP—though I did try to make that somewhat accurate, albeit abstracted and only a subset of the original thread, so I also don’t want to imply that I’m not asserting it.]
From Imaginary!Dai’s perspective, Imaginary!Dai has a question-blob (perhaps a hobby horse—which is fine/good to have), which is about strategy, and deference, and Yudkowsky being deferred to and over-deferred to, and why did that happen, and what bad effects it had, and how to do strategy well individually and as groups, and so on. So then there’s Tsvi’s post, and Imaginary!Dai starts some lines of discussion, relating to his question-blob. Tsvi is replying in the thread, and Imaginary!Dai is continuing the discussion. Tsvi’s sorta going off on slight-tangents, not quite focusing on the interesting/important things, so Imaginary!Dai is somewhat continuing the interesting lines of discussion, while responding to some of Tsvi’s comments in ways that further the interesting parts. Then Tsvi freaks out and bans him.
Ok. So, that would be among my mainline guesses of experiences that Dai was having. I think this possible perspective is empathizable-with, and is mostly fine. It’s also unfortunately perfectly compatible with my description of a charging hobby horse. I don’t defend and endorse all of my behavior in response, but on the question of whether Dai’s behavior had a significant inappropriate pattern, I continue to think yes.
(Whereas I think, and I think that Dai thinks, that the question is relevantly on-topic, because even if everyone in the conversation agrees on the broad outlines of why deference is harmful, they might disagree on the nitty-gritty details of exactly how harmful and exactly why, and litigating the nitty-gritty details of an example that was brought up in the post might help in evaluating the post’s thesis.)
I definitely agree that we might (and in fact do) disagree about various details like that, and that this is in general an important topic.
I’m not sure I understand how it’s on topic though. The topic of the post is “If you’re going to defer, how can you alleviate the problems with that?”.
I think it would be on topic to say “We should have been using a different procedure to choose who to defer to” or “We should have been deferring on different questions” or “We should have deferred in a different pattern, e.g. to a larger group of people or with different of us deferring to a wider range of single people”. It could also be well on-topic to say “We should have been deferring to someone else”, e.g. because Yudkowsky was visibly not the best strategic AGI X-derisk thinker; and so it could be on-topic to discuss “Was Yudkowsky the best AGI X-derisk strategic thinker?” to discuss whether there was a mistaken choice in deferee.
But note that, on that thread, Dai was not AFAICT arguing “We should have deferred to someone else”. He was arguing that we should have deferred less overall. (Which IMO is technically off-topic, though of course quite adjacent. Which is fine/good to discuss. In my first reply to him, I did engage on that question.) He sort of kept discussing “Was Yudkowsky the best AGI X-derisk strategic thinker?”—except AFAICT he completely ignored my initial response “But it seems strange to be counting down...” on that topic, so it was unclear to me whether he was trying to discuss that topic with me.
Anyway:
You wrote earlier:
But it really seems like you do have a significant disagreement with Dai about the extent to which deference to Yudkowsky was justified.
Plausibly! I think we both think he was over-deferred to, probably by “a lot”. But plausibly we have different “a lot”s.
You even seem to ridicule Dai for this (“And then you’re like ‘Ha. Why not just not defer?’”). This seems like a real and substantive disagreement, not a hallucination on Dai’s part.
On my interpretation, my statements make it pretty clear that I think:
People in general defer a lot, and this is quite bad (and implicitly therefore they should defer less in total).
But, people have to defer a lot due to cognitive costs and the complexity of the world (so the solution can’t just be “don’t defer”, or even “don’t defer in your field”, though it can and IMO should be “strive to un-defer on as many key questions as feasible in your field”).
So, geniune question, what did Dai mean by
By saying that he was the best strategic thinker, it seems like you’re trying to justify deferring to him on strategy (why not do that if he is actually the best), while also trying to figure out how to defer “gracefully”, whereas I’m questioning whether it made sense to defer to him at all,
The two natural interpretations I can think of are:
You, Tsvi, by saying he’s the best strategist, are justifying that people, instead of un-deferring-to Yudkowsky when they can, should continue deferring to him.
Insofar as people were deferring, they should have defered not to him at all (but by implication, someone else instead).
I believe I would have ruled out 2, since at that point it did not seem to me that he was putting forth some other candidate as an alternate deferree, preferable to Yudkowsky.
So I interpreted 1, which is why I flew off the handle (which I’m sorry about), given that I’d repeatedly stated that I was not arguing in favor of deferring more than you have to.
(In fact, my actual intentions in bringing up Yudkowsky, was pretty off-hand; right now I think my original intention in even adding the phrase “being the best...” was just to descriptively explain why he was deferred to so much, though I state here that it was for emphasis, which is also plausible.)
Separately, as I mentioned above the question of “how, and how much, are we actually forced to defer” is important and interesting, and maybe technically off-topic or something but a good thread and something I engaged with in my first response.
Sorry, I think I can do better with a little more wordcount and effort.
I think that you think that the reason Dai is wrong to implicitly ask “Why not just not defer in this case?” is because you think that’s not relevantly on-topic for a post about how to mitigate harms from deference by an author who has established that he understands why deference is harmful, because you think that the implied question falsely presupposes that the post author is not aware of why deference is harmful.
(Whereas I think, and I think that Dai thinks, that the question is relevantly on-topic, because even if everyone in the conversation agrees on the broad outlines of why deference is harmful, they might disagree on the nitty-gritty details of exactly how harmful and exactly why, and litigating the nitty-gritty details of an example that was brought up in the post might help in evaluating the post’s thesis.)
Is that closer?
Thank you. IDK if it’s closer, it’s still quite incorrect, but I greatly appreciate the effort.
[Edit: Just to state this: I think Dai was coming from good intentions, e.g. to think about important and interesting things. On reflection: I stand by something being unhelpful about the behavior pattern that I’m claiming I have engaged in, that you have engaged in, and that I claim Dai has engaged in; but also, I don’t think it’s some egregious transgression; I think I overreacted and I would like figure out how to handle discussion more gracefully and non-antagonistically; I value open thoughtful critical discussion and am grateful when people are interested in discussing things related to my writing. I also sorta meant this OP as being somewhat playful, but also I was annoyed, so it might have been more antagonizing than ideal.]
[Edit 2: I’m going to go through some of the points in more detail, because it seems like I haven’t successfully explained the pattern clearly enough yet, so I want to try to make things clearer to Zack. But, I don’t want this detailed wall of text to imply that the situation is some big deal; the behavior I’m discussing is not that bad even if you completely buy my perspective; I don’t mean to harp on it, I just want to answer Zack’s questions.]
I do not think Dai is wrong to ask that question. I think it’s a good and important question.
I don’t think the question was super inexplicit. In his first comment, the last sentence was “In other words, if you were going to spend your career on AI x-safety, of course you could have become an expert on these questions first.”. I followed up with a response to that.
I would say that the fact that his first comment quoted my statement about “Yudkowsky = best AGI X-derisk strategist”, and most of the words were arguing against that, confused the topic. I thought that literal statement was what he wanted to argue against, because he quoted it. Subsequently he did not seem to engage in that specific question, instead starting to develop a different question about different aspects of being a strategic thinker (which is also interesting, but even more off-topic, which is fine, but also confusing).
I do think it’s off-topic, but again, being off-topic is fine. I think it’s polite to acknowledge this, e.g. sometimes if I’m responding to something other than one of the main threads of a post I will (or think I ought to even if I don’t) say “Nitpick:” or “Off-topic, but:” or “I didn’t read the post carefully, just responding to [quote]:”. But being maybe mildly impolite in this way is not by itself the issue I have with that thread (though it is one element of the pattern I describe in the OP).
Let me venture a description of the events from Dai’s perspective. I will write Imaginary!Dai to emphasize that this is not Dai’s perspective, but my attempt to guess a plausible-to-me way his experience might have been. [I should maybe have similarly used Straw!Dai in the dialogue in the OP—though I did try to make that somewhat accurate, albeit abstracted and only a subset of the original thread, so I also don’t want to imply that I’m not asserting it.]
Ok. So, that would be among my mainline guesses of experiences that Dai was having. I think this possible perspective is empathizable-with, and is mostly fine. It’s also unfortunately perfectly compatible with my description of a charging hobby horse. I don’t defend and endorse all of my behavior in response, but on the question of whether Dai’s behavior had a significant inappropriate pattern, I continue to think yes.
I definitely agree that we might (and in fact do) disagree about various details like that, and that this is in general an important topic.
I’m not sure I understand how it’s on topic though. The topic of the post is “If you’re going to defer, how can you alleviate the problems with that?”.
I think it would be on topic to say “We should have been using a different procedure to choose who to defer to” or “We should have been deferring on different questions” or “We should have deferred in a different pattern, e.g. to a larger group of people or with different of us deferring to a wider range of single people”. It could also be well on-topic to say “We should have been deferring to someone else”, e.g. because Yudkowsky was visibly not the best strategic AGI X-derisk thinker; and so it could be on-topic to discuss “Was Yudkowsky the best AGI X-derisk strategic thinker?” to discuss whether there was a mistaken choice in deferee.
But note that, on that thread, Dai was not AFAICT arguing “We should have deferred to someone else”. He was arguing that we should have deferred less overall. (Which IMO is technically off-topic, though of course quite adjacent. Which is fine/good to discuss. In my first reply to him, I did engage on that question.) He sort of kept discussing “Was Yudkowsky the best AGI X-derisk strategic thinker?”—except AFAICT he completely ignored my initial response “But it seems strange to be counting down...” on that topic, so it was unclear to me whether he was trying to discuss that topic with me.
Anyway:
You wrote earlier:
Plausibly! I think we both think he was over-deferred to, probably by “a lot”. But plausibly we have different “a lot”s.
On my interpretation, my statements make it pretty clear that I think:
People in general defer a lot, and this is quite bad (and implicitly therefore they should defer less in total).
But, people have to defer a lot due to cognitive costs and the complexity of the world (so the solution can’t just be “don’t defer”, or even “don’t defer in your field”, though it can and IMO should be “strive to un-defer on as many key questions as feasible in your field”).
So, geniune question, what did Dai mean by
The two natural interpretations I can think of are:
You, Tsvi, by saying he’s the best strategist, are justifying that people, instead of un-deferring-to Yudkowsky when they can, should continue deferring to him.
Insofar as people were deferring, they should have defered not to him at all (but by implication, someone else instead).
I believe I would have ruled out 2, since at that point it did not seem to me that he was putting forth some other candidate as an alternate deferree, preferable to Yudkowsky.
So I interpreted 1, which is why I flew off the handle (which I’m sorry about), given that I’d repeatedly stated that I was not arguing in favor of deferring more than you have to.
(In fact, my actual intentions in bringing up Yudkowsky, was pretty off-hand; right now I think my original intention in even adding the phrase “being the best...” was just to descriptively explain why he was deferred to so much, though I state here that it was for emphasis, which is also plausible.)
Separately, as I mentioned above the question of “how, and how much, are we actually forced to defer” is important and interesting, and maybe technically off-topic or something but a good thread and something I engaged with in my first response.