Well, then I would think we need have a very careful look at systems which define arbitrary minimum and maximum utility values. Do you see no way of making that work either?
The problem with that is that the utility function is not up for grabs. If some tragedy is really so terrible that even a 1/3^^^3 chance of it occurring is worse that, say, losing $5, then that is how morality actually works. You can’t just change your morality because it’s too hard to implement.
If we actually do have bounded utility functions, then Solomonoff induction would allow us to assign expected utilities. There could still be scenarios like Pascal’s mugging if the bound is high enough, so, depending on the size of the bound, it might not add up to exactly what we would expect, but it would be much less of a problem than with unbounded utilities.
Do we give up on symbolic logic?
It’s way too early for that. We know that we have to change at least something from the model described in de Blanc’s paper, but the limitations of this model just show that we don’t know how to find a solution, not that there is no solution. One thing that might help is to better understand our utility functions, and in particular the how they handle infinities, since we currently have tonsofproblems with unbounded and infinite things.
The problem with that is that the utility function is not up for grabs. If some tragedy is really so terrible that even a 1/3^^^3 chance of it occurring is worse that, say, losing $5, then that is how morality actually works. You can’t just change your morality because it’s too hard to implement.
If we actually do have bounded utility functions, then Solomonoff induction would allow us to assign expected utilities. There could still be scenarios like Pascal’s mugging if the bound is high enough, so, depending on the size of the bound, it might not add up to exactly what we would expect, but it would be much less of a problem than with unbounded utilities.
It’s way too early for that. We know that we have to change at least something from the model described in de Blanc’s paper, but the limitations of this model just show that we don’t know how to find a solution, not that there is no solution. One thing that might help is to better understand our utility functions, and in particular the how they handle infinities, since we currently have tons of problems with unbounded and infinite things.