Suppose that the AIs LACK welfare. Then how would we punt on AI GOVERNANCE which is supposed to establish what mankind or whoever concentrates power will ORDER the AIs to do and how to ensure that power isn’t concentrated?
I don’t think those questions depend on one another. We should govern powerful AI systems whether or not they possess welfare. We already regulate many technologies that have no welfare at all. My point is about a different question: when, if ever, does an AI become the kind of thing to which governance is owed, rather than merely applied? Capability alone doesn’t answer that.
Suppose that the AIs LACK welfare. Then how would we punt on AI GOVERNANCE which is supposed to establish what mankind or whoever concentrates power will ORDER the AIs to do and how to ensure that power isn’t concentrated?
I don’t think those questions depend on one another. We should govern powerful AI systems whether or not they possess welfare. We already regulate many technologies that have no welfare at all. My point is about a different question: when, if ever, does an AI become the kind of thing to which governance is owed, rather than merely applied? Capability alone doesn’t answer that.