The main question was basically, why do you think AI researchers generally not engage with Less Wrong/MIRI/Eliezer, what are good reasons behind that that should be taken to heart as valuable learning experiences, and what are bullshit reasons that can and should still be addressed, considered challenges to hack.
I just see a massive discrepancy between what is going on in this community here, and the people actually working in AI implementation and policy, they feel like completely separate spheres. I see problems in both spheres, as well as my own sphere (academic philosophy), and immense potential for mutual gain if cooperation and respect were deepened, and would like to merge them, and wonder how.
I do not see the primary challenge in making a good argument as to why this would be good. I see the primary challenge as a social hacking challenge which includes passing tests set by another group you do not agree with.
It may be useful to wonder what brings people to AI research and what brings people to LessWrong/MIRI? I don’t want to pigeonhole people or stereotype but it could simply be the difference between entrepreneurs (market focused personal spheres) and researchers (field focused personal spheres). Yudkowksy in one interview even recommended paid competitions to solve alignment problems. Paid competitions with high dollar amount prizes could incentivize the separate spheres to comingle.
Very intriguing idea, thank you! Both reflecting on how people end up in these places (has me wonder how one might do qualitative and quantitative survey research to tickle that one out...), and the particular solution.
This is a huge practical issue that seems to not get enough thought, and I’m glad you’re thinking about it. I agree with your summary of one way forward. I think there’s another PR front; many educated people outside of the relevant fields are becoming concerned.
It sounds like the ML researchers at that conference are mostly familiar with MIRI style work. And they actually agree with Yudkowsky that it’s a dead end. There’s a newer tradition of safety work focused on deep networks. That’s what you mostly see in the Alignment Forum. And it’s what you see in the safety teams at Deepmind, OpenAI, and Anthropic. And those companies appear to be making more progress than all the academic ML researchers put together.
Agreed on the paragraph size comment. My eyes and brain shy away. Paragraphs I think are supposed to contain roughly one idea, so a one-sentence paragraph is a nice change of pace if it’s an important idea. Your TLDR was great; I think those are better at the top to function as an abstract and tell the reader why they might want to read the whole piece and how to mentally organize it. ADHD is a reason your brain wants to write stream of consciousness, and attention to paragraph structure is a great check on communicating to others in a way that won’t overwhelm their slower brains :)
Thank you for the feedback, and I am sorry. ADHD.
The main question was basically, why do you think AI researchers generally not engage with Less Wrong/MIRI/Eliezer, what are good reasons behind that that should be taken to heart as valuable learning experiences, and what are bullshit reasons that can and should still be addressed, considered challenges to hack.
I just see a massive discrepancy between what is going on in this community here, and the people actually working in AI implementation and policy, they feel like completely separate spheres. I see problems in both spheres, as well as my own sphere (academic philosophy), and immense potential for mutual gain if cooperation and respect were deepened, and would like to merge them, and wonder how.
I do not see the primary challenge in making a good argument as to why this would be good. I see the primary challenge as a social hacking challenge which includes passing tests set by another group you do not agree with.
It may be useful to wonder what brings people to AI research and what brings people to LessWrong/MIRI? I don’t want to pigeonhole people or stereotype but it could simply be the difference between entrepreneurs (market focused personal spheres) and researchers (field focused personal spheres). Yudkowksy in one interview even recommended paid competitions to solve alignment problems. Paid competitions with high dollar amount prizes could incentivize the separate spheres to comingle.
Very intriguing idea, thank you! Both reflecting on how people end up in these places (has me wonder how one might do qualitative and quantitative survey research to tickle that one out...), and the particular solution.
This is a huge practical issue that seems to not get enough thought, and I’m glad you’re thinking about it. I agree with your summary of one way forward. I think there’s another PR front; many educated people outside of the relevant fields are becoming concerned.
It sounds like the ML researchers at that conference are mostly familiar with MIRI style work. And they actually agree with Yudkowsky that it’s a dead end. There’s a newer tradition of safety work focused on deep networks. That’s what you mostly see in the Alignment Forum. And it’s what you see in the safety teams at Deepmind, OpenAI, and Anthropic. And those companies appear to be making more progress than all the academic ML researchers put together.
Agreed on the paragraph size comment. My eyes and brain shy away. Paragraphs I think are supposed to contain roughly one idea, so a one-sentence paragraph is a nice change of pace if it’s an important idea. Your TLDR was great; I think those are better at the top to function as an abstract and tell the reader why they might want to read the whole piece and how to mentally organize it. ADHD is a reason your brain wants to write stream of consciousness, and attention to paragraph structure is a great check on communicating to others in a way that won’t overwhelm their slower brains :)