High energy ethics and general moral relativity
Utilitarianism sometimes supports weird things: killing lone backpackers for their organs, sacrificing all world’s happiness to one utility monster, creating zillions of humans living on near-subsistence level to maximize total utility, or killing all but a bunch of them to maximize average utility. Also, it supports gay rights, and has been supporting them since 1785, when saying that there’s nothing wrong in having gay sex was pretty much in the same category as saying that there’s nothing wrong in killing backpackers. This makes one wonder: if despite all the disgust towards them few centuries ago, gay rights have been inside the humanity’s coherent extrapolated volition all along, then perhaps our descendants will eventually come to the conclusion that killing the backpacker has been the right choice all along, and only those bullet-biting extremists of our time were getting it right. As a matter of fact, as a friend of mine pointed out, you don’t even need to fast forward few centuries—there are or were already ethical systems actually in use in some cultures (e.g. bushido in pre-Meiji restoration Japan) that are obsessed with honor and survivor’s guilt. They would approve of killing the backpacker or letting them kill themselves—this being an honorable death, and living while letting five other people to die being dishonorable—on non-utilitarian grounds, and actually alieve that this is the right choice. Perhaps they were right all along, and the Western civilization bulldozed through them effectively destroying such culture not because of superior (non-utilitarian) ethics but for any other reason things happened in history. In this case there’s no need in trying to fix utilitarianism, lest it suggest killing backpackers, because it’s not broken—we are—and out descendants will figure that out. In physics we’ve seen this, when an elegant low-Kolmogorov-complexity model predicted that weird things happens on a subatomic level, and we’ve built huge particle accelerators just to confirm—yep, that’s exactly what happens, in spite of all your intuitions. Perhaps smashing utilitarianism with high energy problems only breaks our intuitions, while utilitarianism is just fine.
But let’s talk about relativity. In 1916 Karl Schwarzschild solved the newly discovered Einstein field equations and thus predicted the black holes. It was thought as a mere curiosity and perhaps GIGO at the time, until in 1960s people realized that yes, contra all intuitions, this is in fact a thing. But here’s the thing: they were actually first predicted by John Michell in 1783. You can easily check it: if you substitute the speed of light to the classical formula for escape velocity, you’ll get the Schwarzschild radius. Michell actually knew the radius and mass of the Sun, as well as the gravitational constant precisely enough to get the order of magnitude and the first digit right when providing an example of such object. If we somehow never discovered general relativity, but managed to build good enough telescopes to observe the stars orbiting the emptiness that we now call Sagittarius A*, if would be very tempting to say: “See? We predicted this centuries ago, and however crazy it seemed, we now know it’s true. That’s what happens when you stick to the robust theories, shut up, and calculate—you stay centuries ahead of the curve.”
We now know that Newtonian mechanics aren’t true, although they’re close to truth when you plug in non-astronomical numbers (and even some astronomical). A star 500 times size and the same density as the Sun, however, is very much astronomical. It is only sheer coincidence that in this exact formula relativistic terms work exactly in the way to give the same solution for the escape velocity as the classical mechanics do. It would be enough for Michell to imagine that his dark star rotates—a thing that Newtonian mechanics say doesn’t matter, although it does—to change the category of this prediction from “miraculously correct” to “expectedly incorrect”. It doesn’t mean that Newtonian mechanics weren’t a breakthrough, better than any single theory existing at the time. But it does mean that it would be premature to people in pre-relativity era to invest into building a starship designed to go ten times the speed of light even if they could—although that’s where “shut up and calculate” could lead them.
And that’s where I think we are with utilitarianism. It’s very good. It’s more or less reliably better than anything else. And it managed to make ethical predictions so far fetched (funny enough, about as far fetched as the prediction of dark stars) that it’s tempting to conclude that the only reason why it keeps making crazy predictions is that we haven’t yet realized they’re not crazy. But we live in the world where Sagittarius A* was discovered, and general relativity wasn’t. The actual 42-ish ethical system will probably converge to utilitarianism when you plug in non-extreme numbers (small numbers of people, non-permanent risks and gains, non-taboo topics). But just because it converged to utilitarianism on one taboo (at the time) topic, and made utilitarianism stay centuries ahead of the moral curve, doesn’t mean it will do the same for others.