I think you’ve correctly identified some instances of a more pervasive problem: any consistent system of ethics is going to have some consequences we don’t intuitively like. That’s because our ethical intuitions aren’t entirely consistent.
If we have any mathematical system of ethics along the lines you describe, solving for its maximum is going to mean maximizing some things and thereby minimizing others. For instance, if every human’s happiness has equal moral weight, we’d end up somehow selecting for people who can be happy using less resources, so we can create more total human happiness.
It seems that almost by definition the best we can do at matching what we like is to fulfill human ethical preferences. That’s roughly the one-vote-per-human rule you discuss. This is different than assigning happiness a worth and solving for it mathematically. People can do whatever they like with their votes, and change them over time.
I haven’t worked through how this logic unfolds over time. Does it more or less work to have current humans vote on everything, including who gets to make how many offspring? If everyone can make as many offspring as they want, the cultures or belief systems that do will quickly dominate all future voting.
This type of votes-for-humans-only doesn’t necessarily have horrific consequences for animals. I think that even if animals don’t have voting rights, humans are likely to do something a fair amount like what they’d want as we become a more mature and better educated species. I would rather be friends with people who care about animal suffering, and I think most others feel the same way. So I’d guess we’d see a post-scarcity future in which animals don’t suffer much; keeping them alive but suffering for aesthetics when we have other options seems obviously monstrous.
There’s lots more interesting discussion to be had on this topic.
I think there’s another issue here. Human moral intuitions are evolved to work well between humans, in a primate troop/village with 50-100 individuals, or perhaps a few such groups allied. Extending these to O(100) million humans in a country or even 8 billion humans on a planet has has worked surprisingly well for us. But once you start to include other sentient creatures, as I show above, a lot of things break down if you try to follow human moral intuitions — which isn’t very surprising, since they’re those are now well out of the distribution they were evolved in. And once you don’t have human moral intuitions guiding and constraining your ethical system design, the design decisions start to get a lot more arbitrary. For any outcome you want it’s generally pretty easy to come up with an ethical system that will make that be the optimum (if nothing else, minus the L2 norm of the difference under some metric between the state of the world and the outcome you want). The challenge is to design something that behaves better than that, and actually gives sensible-looking preference orders, has the right stability properties under perturbations, and works sensibly under a range of conditions.
I think you’ve correctly identified some instances of a more pervasive problem: any consistent system of ethics is going to have some consequences we don’t intuitively like. That’s because our ethical intuitions aren’t entirely consistent.
If we have any mathematical system of ethics along the lines you describe, solving for its maximum is going to mean maximizing some things and thereby minimizing others. For instance, if every human’s happiness has equal moral weight, we’d end up somehow selecting for people who can be happy using less resources, so we can create more total human happiness.
It seems that almost by definition the best we can do at matching what we like is to fulfill human ethical preferences. That’s roughly the one-vote-per-human rule you discuss. This is different than assigning happiness a worth and solving for it mathematically. People can do whatever they like with their votes, and change them over time.
I haven’t worked through how this logic unfolds over time. Does it more or less work to have current humans vote on everything, including who gets to make how many offspring? If everyone can make as many offspring as they want, the cultures or belief systems that do will quickly dominate all future voting.
This type of votes-for-humans-only doesn’t necessarily have horrific consequences for animals. I think that even if animals don’t have voting rights, humans are likely to do something a fair amount like what they’d want as we become a more mature and better educated species. I would rather be friends with people who care about animal suffering, and I think most others feel the same way. So I’d guess we’d see a post-scarcity future in which animals don’t suffer much; keeping them alive but suffering for aesthetics when we have other options seems obviously monstrous.
There’s lots more interesting discussion to be had on this topic.
I think there’s another issue here. Human moral intuitions are evolved to work well between humans, in a primate troop/village with 50-100 individuals, or perhaps a few such groups allied. Extending these to O(100) million humans in a country or even 8 billion humans on a planet has has worked surprisingly well for us. But once you start to include other sentient creatures, as I show above, a lot of things break down if you try to follow human moral intuitions — which isn’t very surprising, since they’re those are now well out of the distribution they were evolved in. And once you don’t have human moral intuitions guiding and constraining your ethical system design, the design decisions start to get a lot more arbitrary. For any outcome you want it’s generally pretty easy to come up with an ethical system that will make that be the optimum (if nothing else, minus the L2 norm of the difference under some metric between the state of the world and the outcome you want). The challenge is to design something that behaves better than that, and actually gives sensible-looking preference orders, has the right stability properties under perturbations, and works sensibly under a range of conditions.