“Smart” implicitly entails “knows the true beliefs”, whereas it doesn’t entail “has the right goals”.
It doesn’t exclude having the right goals, either. You could engineer something whose self-improvement was restricted from affecting its goals. But if that is dangerous, why would you?
“Smart” implicitly entails “knows the true beliefs”, whereas it doesn’t entail “has the right goals”.
It doesn’t exclude having the right goals, either. You could engineer something whose self-improvement was restricted from affecting its goals. But if that is dangerous, why would you?