It would be irrational to believe “it will not come up 5 or 6” because P(P(5 or 6) = 0) = 0, so you know for certain that its false. As you said “Claims about the probability of a given claim being true, helpful as they may be in many cases, are distinct from the claim itself.”
Before taking up any belief (if the situation demands taking up a belief, like in a bet, or living life), a Bayesian would calculate the likelihood of it being true vs the likelihood of it being false, and will favour the higher likelihood. In this case, the likelihood that “it will not come up 5 or 6” is true is 0, so a Bayesian would not take up that position.
Now, you might observe that the belief that “1,2,3 or 4 will come up” is true also holds holds the likelihood of zero. In the case of a dice role, any statement of this form will be false, so a Bayesian will take up beliefs that talk probabilities and not certainties . (As Bigjeff explains, “At the most basic level, the difference between Bayesian reasoning and traditional rationalism is a Bayesian only thinks in terms in likelihoods”)

Ofcourse, one can always say “I don’t know”, but saying “I don’t know” would have an inferior utility in life than being a Bayesian. So, for example, assume that your life depends on a series of dice rolls. You can take two positions:
1) You say “I believe I don’t know what the outcome would be” on every roll.
2) You bet on every dice roll according to the information you have (in other words, You say “I believe that outcome X has Y chance of turning up”.
Both positions would be of course be agreeable, but the second position would give you a higher payoff in life. Or so Bayesians believe.

It would be irrational to believe “it will not come up 5 or 6” because P(P(5 or 6) = 0) = 0, so you know for certain that its false. As you said “Claims about the probability of a given claim being true, helpful as they may be in many cases, are distinct from the claim itself.” Before taking up any belief (if the situation demands taking up a belief, like in a bet, or living life), a Bayesian would calculate the likelihood of it being true vs the likelihood of it being false, and will favour the higher likelihood. In this case, the likelihood that “it will not come up 5 or 6” is true is 0, so a Bayesian would not take up that position. Now, you might observe that the belief that “1,2,3 or 4 will come up” is true also holds holds the likelihood of zero. In the case of a dice role, any statement of this form will be false, so a Bayesian will take up beliefs that talk probabilities and not certainties . (As Bigjeff explains, “At the most basic level, the difference between Bayesian reasoning and traditional rationalism is a Bayesian only thinks in terms in likelihoods”)

Ofcourse, one can always say “I don’t know”, but saying “I don’t know” would have an inferior utility in life than being a Bayesian. So, for example, assume that your life depends on a series of dice rolls. You can take two positions: 1) You say “I believe I don’t know what the outcome would be” on every roll. 2) You bet on every dice roll according to the information you have (in other words, You say “I believe that outcome X has Y chance of turning up”. Both positions would be of course be agreeable, but the second position would give you a higher payoff in life. Or so Bayesians believe.