Being Moral is an end goal.

Alice wants to go to her best friends party. She has a sore throat, and she usually takes a COVID test when she has a sore throat. But if the test were to come out positive she would feel like she shouldn’t go, and she really really wants to go. Solution—she avoids taking the test, leaving her probably COVID free, and ready to party.

I’ve heard this story in about a thousand variations since COVID restrictions began.

Analysed from a rational perspective this doesn’t seem to make much sense.

Let’s say that Alice thinks she a 10% chance of having COVID. Tests are perfectly accurate. Alice care about other people, so going to a party whilst infected is worth negative 50 utils. Going to a party is otherwise worth 10 utils.

Then expected utility if Alice takes the test is 0.9 * 10 + 0.1 * 0 = 9.

Expected utility if she doesn’t take the test is 0.9 * 10 + 0.1 * −50 = 5.

So taking the test leaves her better off than not taking the test.

This is a special case of the general rule that knowing more information should never be a negative for a rational agent.

So what’s going on here?

The answer is that Alice doesn’t actually care about other people. Going to a party whilst infectious is worth just as many utils to her as going to a party whilst COVID free.

What Alice cares about is being a moral person. And moral people don’t go to parties whilst knowing they’re infectious. But moral people do go to parties whilst they might have an infection. So if she avoids finding out if she infectious she gets a guaranteed 10 utils at 0 cost.

One way to model this is that a moral action is one that somebody who genuinely cares about other people will do. We don’t genuinely care about other people, but we care about doing moral actions. So we make the minimal changes from pure moral action so that we can mostly take moral actions, whilst still mostly just doing whatever we want. By making the slightly immoral action of not doing a COVID test, we avoid having to make the very immoral action of going to a party whilst infected.

Note that I don’t want to claim people don’t care about other people at all. I’m reducing the motives in this particular case to a simplified human with only a single motivation, and exploring that particular facet of our motives, but as ever, people are complex and other scenarios can best be explained by assuming that people do genuinely care for others.