But there are many bad outcomes to avert, which makes pausing AI—as difficult as that would be—easier than solving all the post-alignment problems in a short time span.
I agree this is a good reason to support pausing AI, and suspect that more people would support pausing if they realized/understood this, but I’m not sure… You can see here where I gave a version of this argument to @wdmacaskill but did not get a response, and here where I reposted the same argument as a LW shortform, but it didn’t get much uptake (or pushback).
Yeah, it generally confuses me why people who talk about post-alignment problems[1] tend not to support pause, and people who support pause tend not to talk much about these problems.
Fwiw I think a bit about post alignment problems and think we should we preparing to pause / slow down for this kind of reason! Compared to standard pause supporters, I’d probably put more emphasis on avoiding concentration of power when we do it and doing it when ai can significantly accelerate efforts to solve these problems
>doing it when ai can significantly accelerate efforts to solve these problems
I just wrote a comment on EAF arguing for pausing now instead of, or in addition to, later, from the perspective of Forethought’s “Better Futures” (which is similar to Michael’s “post-alignment problems”). Maybe you can chime in there if you’re interested in discussing this?
Yeah, it generally confuses me why people who talk about post-alignment problems[1] tend not to support pause, and people who support pause tend not to talk much about these problems.
I have decided that my new hobbyhorse is getting people who talk about post-alignment problems to change their minds on pausing, or at minimum at least engage with the possibility instead of ignoring it. I’m actually not super confident that pause advocacy is the best move on the margin but at least I want people to consider it more seriously.
RE the name, my first draft called them “non-alignment problems”, but a reviewer said this makes it sounds like “the problem of AI not being aligned”. I spent a long time thinking about names and couldn’t come up with anything satisfying. “non-alignment AI x-risks” is too long IMO.
I think of post-alignment problems as “after” alignment in the sense that if you mess up ASI, then the problem that kills you first is misalignment.
I agree this is a good reason to support pausing AI, and suspect that more people would support pausing if they realized/understood this, but I’m not sure… You can see here where I gave a version of this argument to @wdmacaskill but did not get a response, and here where I reposted the same argument as a LW shortform, but it didn’t get much uptake (or pushback).
Yeah, it generally confuses me why people who talk about post-alignment problems[1] tend not to support pause, and people who support pause tend not to talk much about these problems.
Perhaps “non-alignment AI x-risks” would be more accurate to not imply that they are after alignment in some sense? Or maybe “ex-alignment problems”?
Fwiw I think a bit about post alignment problems and think we should we preparing to pause / slow down for this kind of reason! Compared to standard pause supporters, I’d probably put more emphasis on avoiding concentration of power when we do it and doing it when ai can significantly accelerate efforts to solve these problems
>doing it when ai can significantly accelerate efforts to solve these problems
I just wrote a comment on EAF arguing for pausing now instead of, or in addition to, later, from the perspective of Forethought’s “Better Futures” (which is similar to Michael’s “post-alignment problems”). Maybe you can chime in there if you’re interested in discussing this?
I have decided that my new hobbyhorse is getting people who talk about post-alignment problems to change their minds on pausing, or at minimum at least engage with the possibility instead of ignoring it. I’m actually not super confident that pause advocacy is the best move on the margin but at least I want people to consider it more seriously.
RE the name, my first draft called them “non-alignment problems”, but a reviewer said this makes it sounds like “the problem of AI not being aligned”. I spent a long time thinking about names and couldn’t come up with anything satisfying. “non-alignment AI x-risks” is too long IMO.
I think of post-alignment problems as “after” alignment in the sense that if you mess up ASI, then the problem that kills you first is misalignment.