any chance you’d be willing to go into more detail? it sounds like you’re saying unaligned relative to human baseline. I don’t actually think I disagree a priori, I do think people who seek to have high agency have a tendency to end up misaligned with those around them and harming them, for basically exactly the same reasons as any ai that seeks to have high agency. it’s not consistent, though, as far as I can tell; some people successfully decide/reach-internal-consensus to have high agency towards creating moral good and then proceed to successfully apply that decision to the world. The only way to know if this is happening is to do it oneself, of course, and that’s not always easy. Nobody can force you to be moral, so it’s up to you to do it, and there are a lot of ways one can mess it up, notably such as by accepting instructions or worldview that sound moral but aren’t, and claims that someone knows what’s moral often come packaged with claims of exactly the type you and I are making here; “you’re wrong and should change”, after all, is a key way people get people to do things.
any chance you’d be willing to go into more detail? it sounds like you’re saying unaligned relative to human baseline. I don’t actually think I disagree a priori, I do think people who seek to have high agency have a tendency to end up misaligned with those around them and harming them, for basically exactly the same reasons as any ai that seeks to have high agency. it’s not consistent, though, as far as I can tell; some people successfully decide/reach-internal-consensus to have high agency towards creating moral good and then proceed to successfully apply that decision to the world. The only way to know if this is happening is to do it oneself, of course, and that’s not always easy. Nobody can force you to be moral, so it’s up to you to do it, and there are a lot of ways one can mess it up, notably such as by accepting instructions or worldview that sound moral but aren’t, and claims that someone knows what’s moral often come packaged with claims of exactly the type you and I are making here; “you’re wrong and should change”, after all, is a key way people get people to do things.