Prevent the creation of any kaiju (moratorium on some types of AI research)
But when option one is proposed, people say that it has proved to be probably infeasible, and when option two is proposed, people say that the political and economic systems at present cannot be shifted to make such a moratorium happen effectively. If you really believed that alignment was likely impossible, you would advocate for #2 even if you didn’t think it was likely to happen due to politics. The pessimism here just doesn’t make any sense to me.
I think people here are uncomfortable advocating for political solutions either because of their views of politics or their comfort level with it.
You don’t have to believe that alignment is impossible to conclude that you should advocate for a political/governmental solution. All you have to believe is that the probability of x-risk from AGI is reasonably high and the probably of alignment working to prevent it it not reasonably high. That seems to describe the belief of most of those on LessWrong.
I personally do not consider (1) to have been “proved to be probably infeasible”. MIRI had, like, a dozen people working on it for a decade, which just isn’t that much in the scheme of things. And even then, most of those people were not working directly on the core problems for most of that time. The evidence-of-hardness-from-people-trying-and-failing for alignment is not even remotely in the league of, say, P vs NP.
(The evidence-of-hardness-from-people-trying-and-failing is enough that the first clever idea any given person has won’t work, though. Or the fifth idea. Also, just counting MIRI’s research understates the difficulty somewhat, since lots of people worked on various aspects of agent foudations over the past century.)
Certainly I expect that (1) is orders of magnitude easier than (2).
This seems like a misunderstanding of “overseer”-type proposals. ~Nobody thinks alignment is impossible; the rejection is the idea of using unaligned AGIs (or aligned-because-they’re-insufficiently-powerful AGIs) to reliably “contain” another unaligned AGI.
So either we:
Create a kaiju we can trust (alignment)
Prevent the creation of any kaiju (moratorium on some types of AI research)
But when option one is proposed, people say that it has proved to be probably infeasible, and when option two is proposed, people say that the political and economic systems at present cannot be shifted to make such a moratorium happen effectively. If you really believed that alignment was likely impossible, you would advocate for #2 even if you didn’t think it was likely to happen due to politics. The pessimism here just doesn’t make any sense to me.
I think people here are uncomfortable advocating for political solutions either because of their views of politics or their comfort level with it.
You don’t have to believe that alignment is impossible to conclude that you should advocate for a political/governmental solution. All you have to believe is that the probability of x-risk from AGI is reasonably high and the probably of alignment working to prevent it it not reasonably high. That seems to describe the belief of most of those on LessWrong.
I personally do not consider (1) to have been “proved to be probably infeasible”. MIRI had, like, a dozen people working on it for a decade, which just isn’t that much in the scheme of things. And even then, most of those people were not working directly on the core problems for most of that time. The evidence-of-hardness-from-people-trying-and-failing for alignment is not even remotely in the league of, say, P vs NP.
(The evidence-of-hardness-from-people-trying-and-failing is enough that the first clever idea any given person has won’t work, though. Or the fifth idea. Also, just counting MIRI’s research understates the difficulty somewhat, since lots of people worked on various aspects of agent foudations over the past century.)
Certainly I expect that (1) is orders of magnitude easier than (2).
This seems like a misunderstanding of “overseer”-type proposals. ~Nobody thinks alignment is impossible; the rejection is the idea of using unaligned AGIs (or aligned-because-they’re-insufficiently-powerful AGIs) to reliably “contain” another unaligned AGI.