The predominant view on LW seems to be “pure AI capabilities research is bad, because capabilities progress alone doesn’t contribute to alignment progress, and capabilities progress without alignment progress means that we’re doomed”.
I feel like I need to point out that LessWrong seemed mostly neutral to me on the subject of capabilities research until my post asking why people weren’t more negative. I mentioned there that I had literally never heard of someone here suggest the “get people to stop contributing to AI development” strategy informally and I didn’t get any counterexamples. As far as I can tell, people just didn’t talk a whole lot about the question, to the point that it was easy for people like Elon Musk to lose the plot on the whole “AI safety” thing and go start essentially a capabilities company in response to the problem. People are also just talking more about capabilities now in general, because of Eliezer’s doomerism and recent gains giving credence some really short (10-15 year) timelines.
The pessimism post got a lot of upvotes, but it’s only been two months since it was made, so there hasn’t been a lot of time for people with alternative opinions to present them. In addition, as I tried to make clear with a lot of conditional language, that post only makes sense as an argument for slowing down research if you believe AI is probably going to kill everyone. I made it largely as a response to Eliezer’s recent doomerism, and as a pseudo-criticism of how his method of dealing with the problem in practice (namely, doing lots of math and ML research in relative quiet) don’t seem to line up with his words. I share a lot of his beliefs, but if you don’t agree with them, or think AI will probably be the best thing ever, then the post’s conclusions about capabilities research don’t necessarily follow.
Other than that my question to you is similar to what johnswentworth said: to what degree does enhancing the capability of, say, OpenAI’s LSTMs actually give alignment-relevant insight into how it works? There was always a trivial sense in which making AGI will teach us about AGI because when we finally turn it on we’ll learn how we’re doomed. The problem is that those innovations don’t by default tend to lead to better understanding of behavior like this; on the contrary, it tends to make these systems even more complicated and unscrutable from a maths, security, and engineering perspective.
That’s people criticizing OpenAI’s particularly stupid stated philosophy of giving AI to everyone; what I haven’t seen is people criticize capabilities gains or ML researchers in general.
Note also the comments of that post, where people mention that LW has been so negative towards the idea of capabilities progress that multiple ML researchers have been concerned about getting murdered by rationalists and have received literal death threats.
I feel like I need to point out that LessWrong seemed mostly neutral to me on the subject of capabilities research until my post asking why people weren’t more negative. I mentioned there that I had literally never heard of someone here suggest the “get people to stop contributing to AI development” strategy informally and I didn’t get any counterexamples. As far as I can tell, people just didn’t talk a whole lot about the question, to the point that it was easy for people like Elon Musk to lose the plot on the whole “AI safety” thing and go start essentially a capabilities company in response to the problem. People are also just talking more about capabilities now in general, because of Eliezer’s doomerism and recent gains giving credence some really short (10-15 year) timelines.
The pessimism post got a lot of upvotes, but it’s only been two months since it was made, so there hasn’t been a lot of time for people with alternative opinions to present them. In addition, as I tried to make clear with a lot of conditional language, that post only makes sense as an argument for slowing down research if you believe AI is probably going to kill everyone. I made it largely as a response to Eliezer’s recent doomerism, and as a pseudo-criticism of how his method of dealing with the problem in practice (namely, doing lots of math and ML research in relative quiet) don’t seem to line up with his words. I share a lot of his beliefs, but if you don’t agree with them, or think AI will probably be the best thing ever, then the post’s conclusions about capabilities research don’t necessarily follow.
Other than that my question to you is similar to what johnswentworth said: to what degree does enhancing the capability of, say, OpenAI’s LSTMs actually give alignment-relevant insight into how it works? There was always a trivial sense in which making AGI will teach us about AGI because when we finally turn it on we’ll learn how we’re doomed. The problem is that those innovations don’t by default tend to lead to better understanding of behavior like this; on the contrary, it tends to make these systems even more complicated and unscrutable from a maths, security, and engineering perspective.
I’ve seen LW have a negative attitude towards capabilities research a lot longer, e.g.
That’s people criticizing OpenAI’s particularly stupid stated philosophy of giving AI to everyone; what I haven’t seen is people criticize capabilities gains or ML researchers in general.
Note also the comments of that post, where people mention that LW has been so negative towards the idea of capabilities progress that multiple ML researchers have been concerned about getting murdered by rationalists and have received literal death threats.