What p-bothers me (sorry couldn’t resist!) about this approach is that “rightness” nowhere explicitly refers to “others”, i.e. other conscious beings / consciousness-moments. Isn’t there an interesting difference between a heap of eight pebbles (very p-bad) and a human getting tortured (very bad)? Concerning the latter, we can point to that human’s first-person-perspective directly evaluating its current conscious state and concluding that the state is bad, i.e. that the person wants to get the hell out of it. This is a source of disvalue, an unfulfilled “want”—for the “other” concerned—which exists independent from what we, or the pebblesorters, might consider to be “moral”. A heap of pebbles doesn’t do anything like that, there is nothing which could be bad for it, and it seems puzzling why the mere existence of a heap of pebbles would be bad in any meaninful sense, all else being equal. The heap of pebbles is only bad for some being if the being is aware of the heap of pebbles and for whatever reason reacts aversively to it. Whereas being tortured, or suffering (defined as unfulfilled desires) is always bad for the being.
Maybe Eliezer’s account already incorporates this implicitly, assuming that most humans terminally care about others. But as I said, it bothers me that this isn’t made explicit. If it were, I think the quasi-relativist conclusion of this post would be less disturbing. If some heaps of pebbles are p-bad, that needn’t bother us because the pebblesorters don’t care about others, so they’re egoists and not ethical, and even though they use analogous ways to label their terms like “p-ethical” doesn’t imply that they compute their ethics according to the same content-criteria (others matter!) as we do.
If you define “morality” broadly, as maximising values you can end up with that sort of thing. Some would take the attitude that if your definitions covers counterintuitive cases, your definition is too broad.
What p-bothers me (sorry couldn’t resist!) about this approach is that “rightness” nowhere explicitly refers to “others”, i.e. other conscious beings / consciousness-moments. Isn’t there an interesting difference between a heap of eight pebbles (very p-bad) and a human getting tortured (very bad)? Concerning the latter, we can point to that human’s first-person-perspective directly evaluating its current conscious state and concluding that the state is bad, i.e. that the person wants to get the hell out of it. This is a source of disvalue, an unfulfilled “want”—for the “other” concerned—which exists independent from what we, or the pebblesorters, might consider to be “moral”. A heap of pebbles doesn’t do anything like that, there is nothing which could be bad for it, and it seems puzzling why the mere existence of a heap of pebbles would be bad in any meaninful sense, all else being equal. The heap of pebbles is only bad for some being if the being is aware of the heap of pebbles and for whatever reason reacts aversively to it. Whereas being tortured, or suffering (defined as unfulfilled desires) is always bad for the being.
Maybe Eliezer’s account already incorporates this implicitly, assuming that most humans terminally care about others. But as I said, it bothers me that this isn’t made explicit. If it were, I think the quasi-relativist conclusion of this post would be less disturbing. If some heaps of pebbles are p-bad, that needn’t bother us because the pebblesorters don’t care about others, so they’re egoists and not ethical, and even though they use analogous ways to label their terms like “p-ethical” doesn’t imply that they compute their ethics according to the same content-criteria (others matter!) as we do.
If you define “morality” broadly, as maximising values you can end up with that sort of thing. Some would take the attitude that if your definitions covers counterintuitive cases, your definition is too broad.