I do not doubt that the AI alignment community is philosophically homogeneous and that homogeneity hinders the epistemic process. I also agree that it tends to be very consequentialist and that consequentialist ethics should not be considered axiomatic. The AI control problem stems from the unstated assumption that a powerful AI is a consequentialist machine. I think that AI should not, in general, be considered a consequentialist machine.
What do we do about this? I can’t speak directly for the AI alignment community, but I can speak about the rationality community. The rationalist community and the AI alignment community are so closely connected that improvements to one ought to spill into the other.
The simplest thing is to promote actual diversity (especially intellectual diversity). That’s a lot easier to say than to do. This field tends to attract extreme people. Intense intellectual selective pressure promotes intellectual homogeneity. I push back against intellectual homogeneity by writing about Eastern thought.
Chinese readers occasionally message me about how my rationalist writings related to classical literature (such as Sunzi’s The Art of War and the poetry of Li Bai) have made them feel welcome in the rationalist community in a way nothing else has. For my non-Chinese readers, this approach provides an on-boarding process into alien ways of thought.
I do not doubt that the AI alignment community is philosophically homogeneous and that homogeneity hinders the epistemic process. I also agree that it tends to be very consequentialist and that consequentialist ethics should not be considered axiomatic. The AI control problem stems from the unstated assumption that a powerful AI is a consequentialist machine. I think that AI should not, in general, be considered a consequentialist machine.
What do we do about this? I can’t speak directly for the AI alignment community, but I can speak about the rationality community. The rationalist community and the AI alignment community are so closely connected that improvements to one ought to spill into the other.
The simplest thing is to promote actual diversity (especially intellectual diversity). That’s a lot easier to say than to do. This field tends to attract extreme people. Intense intellectual selective pressure promotes intellectual homogeneity. I push back against intellectual homogeneity by writing about Eastern thought.
Chinese readers occasionally message me about how my rationalist writings related to classical literature (such as Sunzi’s The Art of War and the poetry of Li Bai) have made them feel welcome in the rationalist community in a way nothing else has. For my non-Chinese readers, this approach provides an on-boarding process into alien ways of thought.