Even if it does care more about human beings on an individual basis, I think the argument holds (unless the difference in caring is extremely large). Including them in the calculus at all would increase the cost a lot (just considering the sheer number of them, and the logistic challenges of untangling the complex interdependent web of natural ecosystems).
It seems pretty natural for the AI to end up caring more about the preference of humans than other beings, but point taken.
Even if it does care more about human beings on an individual basis, I think the argument holds (unless the difference in caring is extremely large). Including them in the calculus at all would increase the cost a lot (just considering the sheer number of them, and the logistic challenges of untangling the complex interdependent web of natural ecosystems).