I think this needs a bit of refinement, but it might work. Humans have a pretty strong immediacy bias; a greater than 0.1% chance of dying in the next hour really gets our attention. Infinity is way too strong; people do stand their ground on battlefields and such. But certainly you can assign a vast negative utility to that outcome as a practical description of how humans actually think, rather than as an ideal utility function describing how we ought to think.
I think this needs a bit of refinement, but it might work. Humans have a pretty strong immediacy bias; a greater than 0.1% chance of dying in the next hour really gets our attention. Infinity is way too strong; people do stand their ground on battlefields and such. But certainly you can assign a vast negative utility to that outcome as a practical description of how humans actually think, rather than as an ideal utility function describing how we ought to think.