Would it have been better to write a diplomatic formal point that would be more likely to convince those people – or is it more important to give people a world model where they understand this type of “not so smart” reasoning is actually common in frontier labs?
Well, I do think he could stand to be slightly more diplomatic, not enough to not say people are being foolish, but are you seriously saying that Seb fucking Krier is a midwit? like, he’s being a fool, he’s not thinking carefully, these are actions he’s taking, but “midwit” just sounds like yudkowsky has spent too much time on twitter. This isn’t actually the behavior I want yudkowsky to change most, though his abrasive style definitely has something to do with my objection to how he processes others’ claims; I also think being abrasive when necessary is important and good and one should just say what one thinks unless it’s actually unsafe to do so. But I think being brave enough to be abrasive and then, if and only if you are actually not convinced by objections, just keep being abrasive, might be closer.
My actual complaint is actually not centrally about whether he posts here, I guess. The central example of why I think there’s something wrong is that IABIED seems to use more metaphor than it should. His ontology feels out of date. If he’s right, and I sure do think he is, then I wish he was able to explain why he’s right in terms that are more reliably technically insightful.
Idk, maybe he just doesn’t want to accidentally push capabilities forward. I know people like that. I was much more paranoid in that direction than I am now for a long time. It could be his reason and is the main one I’d think was a reflectively good move rather than a result of human limitations. Or maybe he really only thinks of himself as a communicator now. But I’d still like him to be more able to do ontology-level updates without breaking his understanding. I want to see the yud who groks SLT and stuff like that, and sticks around here even when disagreed with
It seems like you want him to do more of everything. That’s not a reasonable request. He’s a communicator now, because he decided technical alignment was too hard.
I disagree with that decision, but primarily because I think he’s doing more harm than good by being so abrasive as the public face of the pause and alignment is hard viewpoints. People should be doing that, just not him.
He can’t keep up on the technical level without spending lots more time on it because he’s human. And reportedly he has chronic fatigue or despair or something, which would be pretty understandable in his position.
Having said that, I agree with you that technical alignment is a worthwhile pursuit even if you do think alignment is hard.
I think we should be recruiting communication specialists to do the public comms part of the project so nerds like the rest of us can shut up and do technical and conceptual work.
Nah I want him to do slightly less of what he does and slightly more of trying to keep up with research, because I think it would make his communication more able to land for technical people. This is not a fully general request, I think he has a specific blind spot about underrating the value of skimming technical work that isn’t immediately obviously relevant or is in the wrong ontology to immediately weigh on what he’s doing. And generally keeping up with subfields that feel like they should produce relevant insights even if they haven’t. Being able to speak their latest language when telling them why one thinks they’re making a mistake.
It is a somewhat general claim, this is just an example. But like, I’d hope for a specific kind of research flavor curiosity to come from being slightly more humble.
My attempt at understanding the type of reactions Eliezer doesn’t like and make him less excited about posting here on lesswrong:
In this text, he elaborates why the AI probably won’t just spare us a few resources to keep going:
https://www.lesswrong.com/posts/F8sfrbPjCQj4KwJqn/the-sun-is-big-but-superintelligences-will-not-spare-earth-a
The top comment – getting 160 karma compared to 218 for the post itself – attacks him over calling people who use “Comparative advantage means humans will keep jobs” midwits: https://www.lesswrong.com/posts/F8sfrbPjCQj4KwJqn/the-sun-is-big-but-superintelligences-will-not-spare-earth-a?commentId=nzLm7giTn8JPD6bTF
Now about a year later, look at this example of a GDM employee making a pretty flawed argument based on CA? Would you agree this is well described as “midwit” behavior overapplying maths? https://www.lesswrong.com/posts/tBr4AtpPmwhgfG4Mw/comparative-advantage-and-ai
Would it have been better to write a diplomatic formal point that would be more likely to convince those people – or is it more important to give people a world model where they understand this type of “not so smart” reasoning is actually common in frontier labs?
Well, I do think he could stand to be slightly more diplomatic, not enough to not say people are being foolish, but are you seriously saying that Seb fucking Krier is a midwit? like, he’s being a fool, he’s not thinking carefully, these are actions he’s taking, but “midwit” just sounds like yudkowsky has spent too much time on twitter. This isn’t actually the behavior I want yudkowsky to change most, though his abrasive style definitely has something to do with my objection to how he processes others’ claims; I also think being abrasive when necessary is important and good and one should just say what one thinks unless it’s actually unsafe to do so. But I think being brave enough to be abrasive and then, if and only if you are actually not convinced by objections, just keep being abrasive, might be closer.
My actual complaint is actually not centrally about whether he posts here, I guess. The central example of why I think there’s something wrong is that IABIED seems to use more metaphor than it should. His ontology feels out of date. If he’s right, and I sure do think he is, then I wish he was able to explain why he’s right in terms that are more reliably technically insightful.
Idk, maybe he just doesn’t want to accidentally push capabilities forward. I know people like that. I was much more paranoid in that direction than I am now for a long time. It could be his reason and is the main one I’d think was a reflectively good move rather than a result of human limitations. Or maybe he really only thinks of himself as a communicator now. But I’d still like him to be more able to do ontology-level updates without breaking his understanding. I want to see the yud who groks SLT and stuff like that, and sticks around here even when disagreed with
It seems like you want him to do more of everything. That’s not a reasonable request. He’s a communicator now, because he decided technical alignment was too hard.
I disagree with that decision, but primarily because I think he’s doing more harm than good by being so abrasive as the public face of the pause and alignment is hard viewpoints. People should be doing that, just not him.
He can’t keep up on the technical level without spending lots more time on it because he’s human. And reportedly he has chronic fatigue or despair or something, which would be pretty understandable in his position.
Having said that, I agree with you that technical alignment is a worthwhile pursuit even if you do think alignment is hard.
I think we should be recruiting communication specialists to do the public comms part of the project so nerds like the rest of us can shut up and do technical and conceptual work.
Nah I want him to do slightly less of what he does and slightly more of trying to keep up with research, because I think it would make his communication more able to land for technical people. This is not a fully general request, I think he has a specific blind spot about underrating the value of skimming technical work that isn’t immediately obviously relevant or is in the wrong ontology to immediately weigh on what he’s doing. And generally keeping up with subfields that feel like they should produce relevant insights even if they haven’t. Being able to speak their latest language when telling them why one thinks they’re making a mistake.
It is a somewhat general claim, this is just an example. But like, I’d hope for a specific kind of research flavor curiosity to come from being slightly more humble.
Oh I see. Not only do I agree, but I think this would actually get upvotes.
Yeah I probably wouldn’t have included the birds and stones metaphor if it was up to me and would have just explained the idea