I agree. The concept of “belief” comes apart into different notions in such cases; like, we might explicitly say “I don’t have a belief about that” and we might internally be unable to summon any arguments one way or another, but we might find ourselves making decisions nonetheless.
I do think this is somewhat relevant for humans rather than only AI, though. If we find ourselves paralyzed and unable to act because we are unable to form a belief, we will end up doing nothing, which in many cases will be worse that things we would have done had we assigned any probability at all. Needing to make decisions is a more powerful justification for needing probabilities than Dutch books are.
I agree. The concept of “belief” comes apart into different notions in such cases; like, we might explicitly say “I don’t have a belief about that” and we might internally be unable to summon any arguments one way or another, but we might find ourselves making decisions nonetheless.
I do think this is somewhat relevant for humans rather than only AI, though. If we find ourselves paralyzed and unable to act because we are unable to form a belief, we will end up doing nothing, which in many cases will be worse that things we would have done had we assigned any probability at all. Needing to make decisions is a more powerful justification for needing probabilities than Dutch books are.