My conclusion: Let’s start the meme that Alignment (the technical problem) is fundamentally impossible (maybe it is? why think you can control something supposedly smarter than you?) and that you will definitely kill yourself if you get to the point where finding a solution to Alignment is what could keep you alive. Pull a Warhammer 40k, start banning machine learning, and for that matter, maybe computers (above some level of performance) and software. This would put more humans in the loop for the same tasks we have now, which offers more opportunities to find problems with the process than how a human right now can program 30 lines of C++, have it LGTM’d by one other person at Google and then have those lines of code be used billions of time, per the input of two humans, ever.
(This meme hasn’t undergone sufficient evolution, feel free to attack with countermemes and supporting memes until it evolves into one powerful enough to take over, and delay the death of the world)
“MIRI walked down this road, a faithful scout, trying to solve future problems before they’re relevant. They’re smart, they’re resourceful, they made noise to get other people to look at the problem. They don’t see a solution in sight. If we don’t move now, the train will run us over. There is no technical solution to alignment, just political solutions—just like there’s no technical solution to nuclear war, just treaties and individuals like Petrov doing their best to avoid total annihilation.”
My conclusion: Let’s start the meme that Alignment (the technical problem) is fundamentally impossible (maybe it is? why think you can control something supposedly smarter than you?) and that you will definitely kill yourself if you get to the point where finding a solution to Alignment is what could keep you alive. Pull a Warhammer 40k, start banning machine learning, and for that matter, maybe computers (above some level of performance) and software. This would put more humans in the loop for the same tasks we have now, which offers more opportunities to find problems with the process than how a human right now can program 30 lines of C++, have it LGTM’d by one other person at Google and then have those lines of code be used billions of time, per the input of two humans, ever.
(This meme hasn’t undergone sufficient evolution, feel free to attack with countermemes and supporting memes until it evolves into one powerful enough to take over, and delay the death of the world)
“MIRI walked down this road, a faithful scout, trying to solve future problems before they’re relevant. They’re smart, they’re resourceful, they made noise to get other people to look at the problem. They don’t see a solution in sight. If we don’t move now, the train will run us over. There is no technical solution to alignment, just political solutions—just like there’s no technical solution to nuclear war, just treaties and individuals like Petrov doing their best to avoid total annihilation.”