“shaped by their values” != “aligned with their values”. I think Stuart is saying not that China will solve the alignment problem, but that they won’t be interested in solving it because they’re focused on expanding capabilities, and translating a book won’t change that.
If so, I think he’s wrong here. The book may lead them to realize that unaligned AGI doesn’t actually constitute an improvement in capabilities. It’s the creation of a new enemy. A bridge that might fall down is not a useful bridge and a successful military power, informed of that, wouldn’t want to build it.
It’s in no party’s interests to create AGI that isn’t aligned with at least the people overseeing the research project.
An AGI aligned with a few living humans is generally going to lead to better outcomes than an AGI aligned with nobody at all, there is enough shared, to know that, and no one coherently extrapolated is as crass or parochial as the people we are now. Alignment theory should be promoted with every party.
If you understand that there’s an alignment problem then “shaped by their values” = “aligned with their values”. That’s especially true in a country that has a strong central leadership.
Design specs very but they all include an AGI that actually values human life which is the key AI safety consideration and why it’s desireable to get the book translated.
-
Why? Why should we assume China can simply solve the alignment problem and the AI will follow their values?
“shaped by their values” != “aligned with their values”. I think Stuart is saying not that China will solve the alignment problem, but that they won’t be interested in solving it because they’re focused on expanding capabilities, and translating a book won’t change that.
If so, I think he’s wrong here. The book may lead them to realize that unaligned AGI doesn’t actually constitute an improvement in capabilities. It’s the creation of a new enemy. A bridge that might fall down is not a useful bridge and a successful military power, informed of that, wouldn’t want to build it.
It’s in no party’s interests to create AGI that isn’t aligned with at least the people overseeing the research project.
An AGI aligned with a few living humans is generally going to lead to better outcomes than an AGI aligned with nobody at all, there is enough shared, to know that, and no one coherently extrapolated is as crass or parochial as the people we are now. Alignment theory should be promoted with every party.
If you understand that there’s an alignment problem then “shaped by their values” = “aligned with their values”. That’s especially true in a country that has a strong central leadership.
-
-
Design specs very but they all include an AGI that actually values human life which is the key AI safety consideration and why it’s desireable to get the book translated.