You are importantly sliding from one point to another, and this is not a topic where you can afford to do that. You can’t just tally up the markers that sort of vibe towards “how dangerous is it?” and get an answer about what to do. The arguments are individually true, or false, and what sort of world we live in depends on which specific combination of arguments are true, or false.
If it turns out there is no political will for a shut down or controlled takeoff, then we can’t have a shut down or controlled takeoff. (But that doesn’t change whether AI is likely to FOOM, or whether alignment is easy/hard)
If AI Fooms suddenly, a lot of AI alignment techniques will probably break at once. If things are gradual, smaller things may break 1-2 at a time, and maybe we get warning shots, and this buys us time. But, there’s still the question of what to do with that time.
If alignment is easy, then a reasonable plan is “get everyone to slow down for a couple years so we can do the obvious safety things, just less rushed.” If alignment is hard, that won’t work, you actually need a radically different paradigm of AI development to have any chance of not killing everyone – you may need a lot of time to figure out something new.
if warning shots are possible, a lot of EY’s arguments don’t hold as straightforwardly
None of IABIED’s arguments had to do with “are warning shots possible?”, but even if they did, it is a logical fallacy to say “warning shots are possible, EY arguments arguments are less valid, therefore, this other argument that had nothing to do with warning shots is also invalid.” If you’re doing that kind of sloppy reasoning, then if you get to the warning shot world, if you don’t understand that overwhelmingly powerful superintelligence is qualitatively different from non-overwhelmingly powerful superintelligence, you might think “angle for a 1-2 year slowdown” instead of trying for a longer global moratorium.
(But, repeat, the book doesn’t say anything about whether warning shots)
You are importantly sliding from one point to another, and this is not a topic where you can afford to do that. You can’t just tally up the markers that sort of vibe towards “how dangerous is it?” and get an answer about what to do. The arguments are individually true, or false, and what sort of world we live in depends on which specific combination of arguments are true, or false.
If it turns out there is no political will for a shut down or controlled takeoff, then we can’t have a shut down or controlled takeoff. (But that doesn’t change whether AI is likely to FOOM, or whether alignment is easy/hard)
If AI Fooms suddenly, a lot of AI alignment techniques will probably break at once. If things are gradual, smaller things may break 1-2 at a time, and maybe we get warning shots, and this buys us time. But, there’s still the question of what to do with that time.
If alignment is easy, then a reasonable plan is “get everyone to slow down for a couple years so we can do the obvious safety things, just less rushed.” If alignment is hard, that won’t work, you actually need a radically different paradigm of AI development to have any chance of not killing everyone – you may need a lot of time to figure out something new.
None of IABIED’s arguments had to do with “are warning shots possible?”, but even if they did, it is a logical fallacy to say “warning shots are possible, EY arguments arguments are less valid, therefore, this other argument that had nothing to do with warning shots is also invalid.” If you’re doing that kind of sloppy reasoning, then if you get to the warning shot world, if you don’t understand that overwhelmingly powerful superintelligence is qualitatively different from non-overwhelmingly powerful superintelligence, you might think “angle for a 1-2 year slowdown” instead of trying for a longer global moratorium.
(But, repeat, the book doesn’t say anything about whether warning shots)