I think this sort of consequentialism seems like part of the beliefs of at least one of the Mechanize team, whom one might say were formerly in the AI safety camp, so agree-voted for that reason. However, I just noticed you implied conscious AIs aren’t morally relevant beings, and have to disagree with that, so will remove the agree vote. I think it can be controversial whether AIs are conscious, but if they are conscious of course they’re morally relevant!
Separately, I don’t understand your point about democracy. Can’t that be Sybil-attacked by AIs when they get voting rights after becoming superpersuasive enough to cause that?
I think this sort of consequentialism seems like part of the beliefs of at least one of the Mechanize team, whom one might say were formerly in the AI safety camp, so agree-voted for that reason. However, I just noticed you implied conscious AIs aren’t morally relevant beings, and have to disagree with that, so will remove the agree vote. I think it can be controversial whether AIs are conscious, but if they are conscious of course they’re morally relevant!
Separately, I don’t understand your point about democracy. Can’t that be Sybil-attacked by AIs when they get voting rights after becoming superpersuasive enough to cause that?