Toy example. Suppose every person wanted lots of tasty food for themselves. No one cares in the slightest about other people starving.
In this scenario, everyone is a paperclipper with respect to everyone else, and yet we can all agree that it’s a good idea to build a “feed everyone AI”.
Sometimes you don’t need your values to be in control, you just need them to be included.
Toy example. Suppose every person wanted lots of tasty food for themselves. No one cares in the slightest about other people starving.
In this scenario, everyone is a paperclipper with respect to everyone else, and yet we can all agree that it’s a good idea to build a “feed everyone AI”.
Sometimes you don’t need your values to be in control, you just need them to be included.