Does a universe consisting of a single particle change?
peco
Karma: −2
If you call a utilitarian’s utility function T, then you can pick the dust specks over torture if your utility function is -T.
Since people only make a finite number of decisions in their lifetime, couldn’t their utility function specify every decision independently? (You could have a utility function that is normal except that it says that everything you hear being called 1A is preferable to 1B, and anything you hear being called 2B is preferable to 2A. If this contradicts your normal utility function, this rule is always more important. Even if 2B leads to death, you still choose 2B.)
The utility function would be impossible to come up with in advance, but it exists.
Why can’t the AI just be exactly the same as Hibbard? If Hibbard is flawed in a major way, you could make an AI for every person on Earth (this obviously wouldn’t be practical, but if a few million AI’s are bad the other few billion can deal with them).