Because people want the AI aligned to their CEV, not their immediate intent
I think this is debatable. Many won’t want to change at all.
You say animal suffering will not be solved, because most people seem not to care about animal suffering. If this is the case, why should they listen to you?
Well, they won’t agree, but the animals would agree. This is tyranny of the majority (or the oligarchy, of humans compared to animals). They would dispute the premise. I agree that at some point this is pure exercise of power, some values are just incompatible (e.g. dictatorship of each individual person).
Unless the problem is genuinely unsolvable, in which case we’re just screwed
I think this is debatable. Many won’t want to change at all.
Well, they won’t agree, but the animals would agree. This is tyranny of the majority (or the oligarchy, of humans compared to animals). They would dispute the premise. I agree that at some point this is pure exercise of power, some values are just incompatible (e.g. dictatorship of each individual person).
Yeah, I guess it has this fatal inevitability to it, no? The only way around is prohibition which won’t happen because the incentives are too great. One faction can divert the torrent of history slightly, but not stop it entirely.
I defined what I meant by CEV in a way that doesn’t entail “changing”.