a huge amount of strategic background; as a consequence of being good strategic background, they shifted many people to working on this”
Maybe we should distinguish between being good at thinking about / explaining strategic background, versus being actually good at strategy per se, e.g. picking high-level directions or judging overall approaches? I think he’s good at the former, but people mistakenly deferred to him too much on the latter.
It would make sense that one could be good at one of these and less good at the other, as they require somewhat different skills. In particular I think the former does not require one to be able to think of all of the crucial considerations, or have overall good judgment after taking them all into consideration.
No? They’re all really difficult questions. Even being an expert in one of these would be at least a career. I mean, maybe YOU can, but I can’t, and I definitely can’t do so when I’m just a kid starting to think about how to help with X-derisking.
So Eliezer could become experts in all of them starting from scratch, but you couldn’t even though you could build upon his writings and other people’s? What was/is your theory of why he is so much above you in this regard? (“Being a kid” seems a red herring since Eliezer was pretty young when he did much of his strategic thinking.)
I think he’s good at the former, but people mistakenly deferred to him too much on the latter.
I agree and I said as much, but this also seems like a non sequitur if you’re just trying to say he’s not the best strategic thinker. Someone can be the best and also be “overrated” (or rather, overly deferred-to). I’m saying he is both. The “thinking about / explaining strategic background” is strong evidence of actually being good at strategy. Separately, Yudkowsky is the biggest creator of our chances of social victory, via LW/X-derisking sphere! (I’m not super confident of that, but pretty confident? Any other candidates?) So it’s a bit hard to argue that he didn’t pick that strategic route as well as the technical route! You can’t grade Yudkowsky on his own special curve just for all his various attempts at X-derisking, and then separately grade everyone else.
It would make sense that one could be good at one of these and less good at the other, as they require somewhat different skills. In particular I think the former does not require one to be able to think of all of the crucial considerations, or have overall good judgment after taking them all into consideration.
Ok. I mean, true. I guess someone could suggest alternative candidates, though I’m noticing IDK why to care much about this question.
(I continue to have a sense that you’re misunderstanding what I’m saying, as described earlier, and also not sure what’s interesting about this topic. My bid would be, if there’s something here that seems interesting or important to you, that you would say a bit about what that is and why, as a way of recentering. It seems like you’re trying to drill down into particulars, but you keep being like “So why do you think X?” and I’m like “I don’t think X.”.)
By saying that he was the best strategic thinker, it seems like you’re trying to justify deferring to him on strategy (why not do that if he is actually the best), while also trying to figure out how to defer “gracefully”, whereas I’m questioning whether it made sense to defer to him at all, when you could have taken into account his (and other people’s) writings about strategic background, and then looked for other important considerations and formed your own judgments.
Another thing that interests me is that several of his high-level strategic judgments seemed wrong or questionable to me at the time (as listed in my OP, and I can look up my old posts/comments if that would help), and if it didn’t seem that way to others, I want to understand why. Was Eliezer actually right, given what we knew at the time? Did it require a rare strategic mind to notice his mistakes? Or was it a halo effect, or the effect of Eliezer writing too confidently, or something else, that caused others to have a cognitive blind spot about this?
By saying that he was the best strategic thinker, it seems like you’re trying to justify deferring to him on strategy (why not do that if he is actually the best)
No. You’re totally hallucinating this and also not updating when I’m repeatedly telling you no. It’s also the opposite of the point hammered in by the OP. My entire post is complaining about problems with deferring, and it links a prior post I wrote laying out these problems in detail, and I linked that essay to you again, and I linked several other writings explaining more how I’m against deferring and tell people not to defer repeatedly and in different ways. I bring up Eliezer to say “Look, we deferred to the best strategic thinker, and even though he’s the best strategic thinker, deferring was STILL really bad.”. Since I’ve described how deferring is really bad in several other places, here in THIS post I’m asking, given that we’re going to defer despite its costs, and given that to some extent at the end of the day we do have to defer on many things, what can we do to alleviate some of those problems?
And then you’re like “Ha. Why not just not defer?”.
Since I’ve described how deferring is really bad in several other places, here in THIS post I’m asking, given that we’re going to defer despite its costs, and given that to some extent at the end of the day we do have to defer on many things, what can we do to alleviate some of those problems?
Ok, it looks like part of my motivation for going down this line of thought was based on a misunderstanding. But to be fair, in this post after you asked “What should we have done instead?” with regard to deferring to Eliezer, you didn’t clearly say “we should have not deferred or deferred less”, but instead wrote “We don’t have to stop deferring, to avoid this correlated failure. We just have to say that we’re deferring.” Given that this is a case where many people could have and should have not deferred, this just seems like a bad example to illustrate “given that to some extent at the end of the day we do have to defer on many things, what can we do to alleviate some of those problems?”, leading to the kind of confusion I had.
Also, another part of my motivation is still valid and I think it would be interesting to try to answer why didn’t you (and others) just not defer? Not in a rhetorical sense, but what actually caused this? Was it age as you hinted earlier? Was it just human nature to want to defer to someone? Was it that you were being paid by an organization that Eliezer founded and had very strong influence over? Etc.? And also why didn’t you (and others) notice Eliezer’s strategic mistakes, if that has a different or additional answer?
Also, another part of my motivation is still valid and I think it would be interesting to try to answer why didn’t you (and others) just not defer? Not in a rhetorical sense, but what actually caused this?
Ok, sure, that’s a good question, and also off-topic.
Was it age as you hinted earlier?
Yeah obviously. It’s literally impossible to not defer, all you get to pick is which things you invest in undeferring in what order. I’m exceptionally non-deferential but yeah obviously you have to defer about lots of things.
Was it just human nature to want to defer to someone?
Yes it is also human nature to want to defer. E.g. that’s how you stay synched with your tribe on what stuff matters, how to act, etc.
Was it that you were being paid by an organization that Eliezer founded and had very strong influence over? Etc.?
No, I took being paid as more obligation to not defer.
Anyway, I’m banning you from my posts due to grossly negligent reading comprehension.
The grandparent explains why Dai was confused about your authorial intent, and his comment at the top of the thread is sitting at 31 karma in 15 votes, suggesting that other readers found Dai’s engagement valuable. If that’s grossly negligent reading comprehension, then would you prefer to just not have readers? That is, it seems strange to be counting down from “smart commenters interpret my words in the way I want them to be interpreted” rather than up from “no one reads or comments on my work.”
suggesting that other readers found Dai’s engagement valuable
This may not be a valid inference, or your update may be too strong, given that my comment got a strong upvote early or immediately, which caused it to land in the Popular Comments section of the front page, where others may have further upvoted it in a decontextualized way.
It looks like I’m not actually banned yet, but will disengage for now to respect Tsvi’s wishes/feelings. Thought I should correct the record on the above first, as I’m probably the only person who could (due to seeing the strong upvote and the resulting position in Popular Comments).
I have banned you from my posts, but my guess is that you’re still allowed to post on existing comment threads with you involved, or something like. I’m happy for you to comment on anything that the LW interface allows you to comment on. [ETA: actually I hadn’t hit “submit” on the ban; I’ve done that now, so Wei Dai might no longer be able to reply on this post at all.]
Possibly I’ll unban you some time in the future (not that anyone cares too much, I presume). But like, this comment thread is kinda wild from my perspective. My current understanding is that you “went down some line of questioning” based on a misunderstanding, but did not state what your line of questioning was and also ignored anything in my responses that wasn’t furthering your “line of questioning” including stuff that was correcting your misunderstanding. Which is pretty anti-helpful.
Are you wanting to say “I, Wei Dai, am a better strategic thinker on AGI X-derisking than Yudkowsky.”? That’s a perfectly fine thing to say IMO, but of course you should understand that most people (me included) wouldn’t by default have the context to believe that.
Maybe we should distinguish between being good at thinking about / explaining strategic background, versus being actually good at strategy per se, e.g. picking high-level directions or judging overall approaches? I think he’s good at the former, but people mistakenly deferred to him too much on the latter.
It would make sense that one could be good at one of these and less good at the other, as they require somewhat different skills. In particular I think the former does not require one to be able to think of all of the crucial considerations, or have overall good judgment after taking them all into consideration.
So Eliezer could become experts in all of them starting from scratch, but you couldn’t even though you could build upon his writings and other people’s? What was/is your theory of why he is so much above you in this regard? (“Being a kid” seems a red herring since Eliezer was pretty young when he did much of his strategic thinking.)
I agree and I said as much, but this also seems like a non sequitur if you’re just trying to say he’s not the best strategic thinker. Someone can be the best and also be “overrated” (or rather, overly deferred-to). I’m saying he is both. The “thinking about / explaining strategic background” is strong evidence of actually being good at strategy. Separately, Yudkowsky is the biggest creator of our chances of social victory, via LW/X-derisking sphere! (I’m not super confident of that, but pretty confident? Any other candidates?) So it’s a bit hard to argue that he didn’t pick that strategic route as well as the technical route! You can’t grade Yudkowsky on his own special curve just for all his various attempts at X-derisking, and then separately grade everyone else.
Ok. I mean, true. I guess someone could suggest alternative candidates, though I’m noticing IDK why to care much about this question.
(I continue to have a sense that you’re misunderstanding what I’m saying, as described earlier, and also not sure what’s interesting about this topic. My bid would be, if there’s something here that seems interesting or important to you, that you would say a bit about what that is and why, as a way of recentering. It seems like you’re trying to drill down into particulars, but you keep being like “So why do you think X?” and I’m like “I don’t think X.”.)
By saying that he was the best strategic thinker, it seems like you’re trying to justify deferring to him on strategy (why not do that if he is actually the best), while also trying to figure out how to defer “gracefully”, whereas I’m questioning whether it made sense to defer to him at all, when you could have taken into account his (and other people’s) writings about strategic background, and then looked for other important considerations and formed your own judgments.
Another thing that interests me is that several of his high-level strategic judgments seemed wrong or questionable to me at the time (as listed in my OP, and I can look up my old posts/comments if that would help), and if it didn’t seem that way to others, I want to understand why. Was Eliezer actually right, given what we knew at the time? Did it require a rare strategic mind to notice his mistakes? Or was it a halo effect, or the effect of Eliezer writing too confidently, or something else, that caused others to have a cognitive blind spot about this?
No. You’re totally hallucinating this and also not updating when I’m repeatedly telling you no. It’s also the opposite of the point hammered in by the OP. My entire post is complaining about problems with deferring, and it links a prior post I wrote laying out these problems in detail, and I linked that essay to you again, and I linked several other writings explaining more how I’m against deferring and tell people not to defer repeatedly and in different ways. I bring up Eliezer to say “Look, we deferred to the best strategic thinker, and even though he’s the best strategic thinker, deferring was STILL really bad.”. Since I’ve described how deferring is really bad in several other places, here in THIS post I’m asking, given that we’re going to defer despite its costs, and given that to some extent at the end of the day we do have to defer on many things, what can we do to alleviate some of those problems?
And then you’re like “Ha. Why not just not defer?”.
A bit blackpilling re/ LW voters. So cowardly, and so wrong.
Ok, it looks like part of my motivation for going down this line of thought was based on a misunderstanding. But to be fair, in this post after you asked “What should we have done instead?” with regard to deferring to Eliezer, you didn’t clearly say “we should have not deferred or deferred less”, but instead wrote “We don’t have to stop deferring, to avoid this correlated failure. We just have to say that we’re deferring.” Given that this is a case where many people could have and should have not deferred, this just seems like a bad example to illustrate “given that to some extent at the end of the day we do have to defer on many things, what can we do to alleviate some of those problems?”, leading to the kind of confusion I had.
Also, another part of my motivation is still valid and I think it would be interesting to try to answer why didn’t you (and others) just not defer? Not in a rhetorical sense, but what actually caused this? Was it age as you hinted earlier? Was it just human nature to want to defer to someone? Was it that you were being paid by an organization that Eliezer founded and had very strong influence over? Etc.? And also why didn’t you (and others) notice Eliezer’s strategic mistakes, if that has a different or additional answer?
Ok, sure, that’s a good question, and also off-topic.
Yeah obviously. It’s literally impossible to not defer, all you get to pick is which things you invest in undeferring in what order. I’m exceptionally non-deferential but yeah obviously you have to defer about lots of things.
Yes it is also human nature to want to defer. E.g. that’s how you stay synched with your tribe on what stuff matters, how to act, etc.
No, I took being paid as more obligation to not defer.
Anyway, I’m banning you from my posts due to grossly negligent reading comprehension.
The grandparent explains why Dai was confused about your authorial intent, and his comment at the top of the thread is sitting at 31 karma in 15 votes, suggesting that other readers found Dai’s engagement valuable. If that’s grossly negligent reading comprehension, then would you prefer to just not have readers? That is, it seems strange to be counting down from “smart commenters interpret my words in the way I want them to be interpreted” rather than up from “no one reads or comments on my work.”
This may not be a valid inference, or your update may be too strong, given that my comment got a strong upvote early or immediately, which caused it to land in the Popular Comments section of the front page, where others may have further upvoted it in a decontextualized way.
It looks like I’m not actually banned yet, but will disengage for now to respect Tsvi’s wishes/feelings. Thought I should correct the record on the above first, as I’m probably the only person who could (due to seeing the strong upvote and the resulting position in Popular Comments).
I have banned you from my posts, but my guess is that you’re still allowed to post on existing comment threads with you involved, or something like. I’m happy for you to comment on anything that the LW interface allows you to comment on. [ETA: actually I hadn’t hit “submit” on the ban; I’ve done that now, so Wei Dai might no longer be able to reply on this post at all.]
Possibly I’ll unban you some time in the future (not that anyone cares too much, I presume). But like, this comment thread is kinda wild from my perspective. My current understanding is that you “went down some line of questioning” based on a misunderstanding, but did not state what your line of questioning was and also ignored anything in my responses that wasn’t furthering your “line of questioning” including stuff that was correcting your misunderstanding. Which is pretty anti-helpful.
Did you read the whole comment thread?
Are you wanting to say “I, Wei Dai, am a better strategic thinker on AGI X-derisking than Yudkowsky.”? That’s a perfectly fine thing to say IMO, but of course you should understand that most people (me included) wouldn’t by default have the context to believe that.