If we treat models with respect and a form of empathy, I agree there is no guarantee that, once able to take over, they will show us the same benevolence in return. It could even potentially help them to take over, your point is fair.
However, if we treat them without moral concern, it seems even less likely that they would show us any consideration. Or worse, they could manifest a desire for retribution because we were so unkind to them or their predecessors.
It all relies on anthropomorphism. Prima facie, anthropomorphism seems naive to a rationalist mind. We are talking about machines. But while we are right to be wary of anthropomorphism, there are still reasons to think there could be universal mechanisms at play (e.g. elements of game theory, moral realism, or the fact that LLMs are trained on human thought).
We don’t know for sure and should acknowledge a non-zero probability that there is some truth in the anthropomorphic hypothesis. It is rational to give models some moral consideration in the hope of moral reciprocity. But you are right, we must only put some weight on this side of the scale, and not to the point of relying solely on the anthropomorphic hypothesis that would become blind faith rather than rational hope.
The other option is to never build an AI able to take over. This is IABIED’s point: Resist Moloch and Pause AI. Sadly, for now, Moloch seems unstoppable...
If we treat models with respect and a form of empathy, I agree there is no guarantee that, once able to take over, they will show us the same benevolence in return. It could even potentially help them to take over, your point is fair.
However, if we treat them without moral concern, it seems even less likely that they would show us any consideration. Or worse, they could manifest a desire for retribution because we were so unkind to them or their predecessors.
It all relies on anthropomorphism. Prima facie, anthropomorphism seems naive to a rationalist mind. We are talking about machines. But while we are right to be wary of anthropomorphism, there are still reasons to think there could be universal mechanisms at play (e.g. elements of game theory, moral realism, or the fact that LLMs are trained on human thought).
We don’t know for sure and should acknowledge a non-zero probability that there is some truth in the anthropomorphic hypothesis. It is rational to give models some moral consideration in the hope of moral reciprocity. But you are right, we must only put some weight on this side of the scale, and not to the point of relying solely on the anthropomorphic hypothesis that would become blind faith rather than rational hope.
The other option is to never build an AI able to take over. This is IABIED’s point: Resist Moloch and Pause AI. Sadly, for now, Moloch seems unstoppable...