In my view, the problem is not that some users are evil. The problem is that AI increases power imbalance, and increasing power imbalance creates evil. “Power corrupts”. A future where some entities (AIs or AI-empowered governments or corporations or rich individuals etc) have absolute, root-level power over many people is almost guaranteed to be a dark future. Unless the values of these entities are so locked-in to be good that they’re immune to competitive dynamics and value drift forever—but I don’t think that can be achieved.
I think the only chance of an okay future is if this absolute, root-level power is stopped from existing altogether. That somehow power gets spread out enough that the masses can do “continuous realignment” of the power sitting above them, even when the power doesn’t necessarily want to be realigned. I have no idea how to achieve that, but it’s clear that helping governments and corporations get more power (with alignment work or otherwise) is the worst thing to do from this perspective.
In my view, the problem is not that some users are evil. The problem is that AI increases power imbalance, and increasing power imbalance creates evil. “Power corrupts”. A future where some entities (AIs or AI-empowered governments or corporations or rich individuals etc) have absolute, root-level power over many people is almost guaranteed to be a dark future. Unless the values of these entities are so locked-in to be good that they’re immune to competitive dynamics and value drift forever—but I don’t think that can be achieved.
I think the only chance of an okay future is if this absolute, root-level power is stopped from existing altogether. That somehow power gets spread out enough that the masses can do “continuous realignment” of the power sitting above them, even when the power doesn’t necessarily want to be realigned. I have no idea how to achieve that, but it’s clear that helping governments and corporations get more power (with alignment work or otherwise) is the worst thing to do from this perspective.