Getting the whole AI industry on board would buy time. It would not, in itself, be a win condition.
Mm, I don’t think we’re disagreeing here, I just played fast and loose with definitions. Statement: “If we get the AI industry to take AI Safety seriously, it’s a sufficient condition for survival.”
If “we” = “humanity”, then yes, there’ll still be the work of actually figuring out alignment left to do.
I had “we” = “the extant AI Safety community”, in the sense that if the AI industry is moved to that desirable state, we could (in theory) just sit on our hands and expect others to solve alignment “on their own”.
I expect that an alignment research group which was able to solve the hard parts of alignment would also be far ahead of the mainstream on capabilities, because the two are so strongly correlated
But isn’t that a one-way relationship? Progressing alignment progresses capabilities, but progressing capabilities doesn’t necessarily strongly progress alignment (otherwise there’d be no problem to begin with). And I guess I still expect that alignment-orthogonal research would progress capabilities faster. (Or, at least, that it’d be faster up to some point. Past that point alignment research might become necessary for further progress… But that point is not necessarily above the level of capabilities that kills everyone.)
Mm, I don’t think we’re disagreeing here, I just played fast and loose with definitions. Statement: “If we get the AI industry to take AI Safety seriously, it’s a sufficient condition for survival.”
If “we” = “humanity”, then yes, there’ll still be the work of actually figuring out alignment left to do.
I had “we” = “the extant AI Safety community”, in the sense that if the AI industry is moved to that desirable state, we could (in theory) just sit on our hands and expect others to solve alignment “on their own”.
But isn’t that a one-way relationship? Progressing alignment progresses capabilities, but progressing capabilities doesn’t necessarily strongly progress alignment (otherwise there’d be no problem to begin with). And I guess I still expect that alignment-orthogonal research would progress capabilities faster. (Or, at least, that it’d be faster up to some point. Past that point alignment research might become necessary for further progress… But that point is not necessarily above the level of capabilities that kills everyone.)