I think it’ll be useful to compare to other cases of arbitrariness. E.g. what prior you start with is arbitrary, so all odds you give are scaled by some arbitrary factor. But this doesn’t necessarily make them meaningless, or mean that you shouldn’t make any decisions based on probabilistic considerations.
I’d break down the arguments about priors into ones based on properties, performance, and emotional recalibration.
Properties would be something like Savage’s axioms—if you find them appealing, then you want to make decisions in a way compatible with probabilistic reasoning. So what if there’s a remaining degree of freedom for the prior—Savage’s theorem still holds, so even as you complain that you have no justification for one prior over another, you should still be acting as if you have one.
Performance is something like Dutch Book arguments, or, more powerfully, Solomonoff’s arguments that a simplicity prior will make only a finite number of mistakes in an approximately computable universe. You build an abstract model of how a reasoning style will perform, and then you justify using that reasoning style by appealing to good modeled performance.
The emotional category is based on the idea that we weight arbitrariness “too heavily” in some sense, and that we need to change our emotional outlook on it. In the case of probabilities, it’s important that the arbitrary component is contained to the prior—few people want to accept total arbitrariness. But when the arbitrariness is contained, it’s more appealing to say something like “This arbitrariness is unavoidable, and that’s okay. To worry that we’re making a ‘wrong choice’ or that any choice here needs a further step of justification is to misunderstand what’s going on. This is about expressing ourselves and doing our best, and it’s genuinely okay to be arbitrary in this way.”
If I was giving my own framing of the unawareness problem (I’m not a big fan of setting up P2 in terms of a black-and-white transition from “has an argument” to “doesn’t have an argument”), I’d probably set it up in terms of the choices and simplifying assumptions we must use to arrive at a model of the far future that our limited minds can actually use to make decisions. How do our modeling choices have to be arbitrary or otherwise unjustified?
So what if there’s a remaining degree of freedom for the prior—Savage’s theorem still holds, so even as you complain that you have no justification for one prior over another, you should still be acting as if you have one.
“Acting as if” I have a prior is compatible with admitting cluelessness. See here, which also discusses Dutch book arguments.
Solomonoff’s arguments that a simplicity prior will make only a finite number of mistakes in an approximately computable universe. You build an abstract model of how a reasoning style will perform, and then you justify using that reasoning style by appealing to good modeled performance
One problem with this is that Solomonoff priors are way too computationally intractable for us. So I don’t see what normative relevance these arguments have for us.
But when the arbitrariness is contained, it’s more appealing to say something like “This arbitrariness is unavoidable, and that’s okay. To worry that we’re making a ‘wrong choice’ or that any choice here needs a further step of justification is to misunderstand what’s going on. This is about expressing ourselves and doing our best, and it’s genuinely okay to be arbitrary in this way.”
When you say the arbitrariness is “unavoidable”, is this an implication of your previous paragraphs, or something else? I think the following response avoids some of the arbitrariness: “I simply don’t think the information I have warrants ‘expecting’ A to be better than B, or vice versa. So, I don’t have reason to c-prefer A or B. I’ll make my decisions on some basis other than c-preferences, or at the very least not tell myself I’m choosing A or B based on the impartial good, when that’s false.”
Pretty interesting.
I think it’ll be useful to compare to other cases of arbitrariness. E.g. what prior you start with is arbitrary, so all odds you give are scaled by some arbitrary factor. But this doesn’t necessarily make them meaningless, or mean that you shouldn’t make any decisions based on probabilistic considerations.
I’d break down the arguments about priors into ones based on properties, performance, and emotional recalibration.
Properties would be something like Savage’s axioms—if you find them appealing, then you want to make decisions in a way compatible with probabilistic reasoning. So what if there’s a remaining degree of freedom for the prior—Savage’s theorem still holds, so even as you complain that you have no justification for one prior over another, you should still be acting as if you have one.
Performance is something like Dutch Book arguments, or, more powerfully, Solomonoff’s arguments that a simplicity prior will make only a finite number of mistakes in an approximately computable universe. You build an abstract model of how a reasoning style will perform, and then you justify using that reasoning style by appealing to good modeled performance.
The emotional category is based on the idea that we weight arbitrariness “too heavily” in some sense, and that we need to change our emotional outlook on it. In the case of probabilities, it’s important that the arbitrary component is contained to the prior—few people want to accept total arbitrariness. But when the arbitrariness is contained, it’s more appealing to say something like “This arbitrariness is unavoidable, and that’s okay. To worry that we’re making a ‘wrong choice’ or that any choice here needs a further step of justification is to misunderstand what’s going on. This is about expressing ourselves and doing our best, and it’s genuinely okay to be arbitrary in this way.”
If I was giving my own framing of the unawareness problem (I’m not a big fan of setting up P2 in terms of a black-and-white transition from “has an argument” to “doesn’t have an argument”), I’d probably set it up in terms of the choices and simplifying assumptions we must use to arrive at a model of the far future that our limited minds can actually use to make decisions. How do our modeling choices have to be arbitrary or otherwise unjustified?
Thanks!
“Acting as if” I have a prior is compatible with admitting cluelessness. See here, which also discusses Dutch book arguments.
One problem with this is that Solomonoff priors are way too computationally intractable for us. So I don’t see what normative relevance these arguments have for us.
When you say the arbitrariness is “unavoidable”, is this an implication of your previous paragraphs, or something else? I think the following response avoids some of the arbitrariness: “I simply don’t think the information I have warrants ‘expecting’ A to be better than B, or vice versa. So, I don’t have reason to c-prefer A or B. I’ll make my decisions on some basis other than c-preferences, or at the very least not tell myself I’m choosing A or B based on the impartial good, when that’s false.”