Yeah, I think this basically goes through. Though, even if we did have the ability to make rule-following AI, that doesn’t mean we’re now safe to go ahead. There are several other hurdles, like finding rules which make things good when superintelligent optimization is applied, and getting good enough goal-content integrity to not miss a step of self modification, plus the various human shaped challenges.
Yeah, I think this basically goes through. Though, even if we did have the ability to make rule-following AI, that doesn’t mean we’re now safe to go ahead. There are several other hurdles, like finding rules which make things good when superintelligent optimization is applied, and getting good enough goal-content integrity to not miss a step of self modification, plus the various human shaped challenges.