This seems to be a popular topic for non-LW sources to link to us over. So in the interests of publicity, entertainment, reduced x-risk, and marginally less risk of anyone ever being tortured, here is a stronger, yet seemingly more benign alternative: the Gentle Judge.
GJ is more intuitive: Fair punishments for intentional refusals to cooperate with Friendly goals. One simple way to determine fair punishment would be to examine the individual’s own perferences, and see how they would punish another person according to their own value system, given the same set of actions and consequences. However, there could also be a ceiling based on what the group of individuals being considered for punishment would believe is fair (ensuring that unusually vindictive outliers don’t end up being punished excessively).
Like Roko’s Basilisk, GJ depends on people imagining it in sufficient detail and being influenced by that to cooperate. Unlike RB, it is not hyper-morally-counterintuitive, so more people would tend to adopt it (all else equal). This means that more people could e.g. donate to MIRI (if that is the best path to GJ’s goals in their subjective estimation). Moreover, if GJ is more likely to be successful, any worlds with RB would actually punish people who fail to promote and/or implement GJ instead of RB.
This seems to be a popular topic for non-LW sources to link to us over. So in the interests of publicity, entertainment, reduced x-risk, and marginally less risk of anyone ever being tortured, here is a stronger, yet seemingly more benign alternative: the Gentle Judge.
GJ is more intuitive: Fair punishments for intentional refusals to cooperate with Friendly goals. One simple way to determine fair punishment would be to examine the individual’s own perferences, and see how they would punish another person according to their own value system, given the same set of actions and consequences. However, there could also be a ceiling based on what the group of individuals being considered for punishment would believe is fair (ensuring that unusually vindictive outliers don’t end up being punished excessively).
Like Roko’s Basilisk, GJ depends on people imagining it in sufficient detail and being influenced by that to cooperate. Unlike RB, it is not hyper-morally-counterintuitive, so more people would tend to adopt it (all else equal). This means that more people could e.g. donate to MIRI (if that is the best path to GJ’s goals in their subjective estimation). Moreover, if GJ is more likely to be successful, any worlds with RB would actually punish people who fail to promote and/or implement GJ instead of RB.