Researcher at the Center on Long-Term Risk. All opinions my own.
Anthony DiGiovanni
Karma: 1,329
one should still support it under precise credences?
I’m saying that Elga’s argument doesn’t tell us to have precise credences in the first place. It only tells us “you should commit to act in a way that avoids sure losses”.
(I generally find LLM-written philosophy critiques overstate various things, which I don’t think are worth the time to engage with. Just briefly replying here to the substantive points.)
“The imprecise credence itself doesn’t guide the action w.r.t. the second bet” does not imply “you shouldn’t have imprecise credences in general”. Elga’s argument doesn’t tell us at all that we should, say, “go with our best guess” about altruistic interventions.
I don’t understand the critique. If C does “dodge the objection”, that’s an important advantage of C! And a commitment is a substantively different move from choosing what to do about bet B once bet A has already been dealt with, so it’s not an ad hoc dodge.
Again, see (1). Whether the commitment (without the imprecise credences) suffices to avoid this particular Dutch book tells us nothing about whether, when we’re making altruistic decisions, we should adopt precise credences and EV-max w.r.t. them. Claude’s point about “adds nothing a sharp state lacks” ignores all the positive epistemic motivations for imprecision I’ve argued for in the sequence.
Following (3): There are positive epistemic motivations for imprecise credences. So yes, Elga is asking the impreciser to override what they consider locally (epistemically) rational.
I’m proposing binding commitments, not plans. As I say, there’s no choice to be made after committing to C and rejecting bet A. So I reject the claim that the commitment response requires “imposing different requirements on choices that are identical in all relevant respects”.
Summary of why I don’t buy diachronic Dutch book arguments
(Using Elga’s Dutch book against imprecise credences as an example, where the agent faces a sequence of two bets A and B.)
Consider the action C: “Commit to accept B if you first reject A.” By “commit”, I mean literally lock your future self out of the “reject B” action (if you reject A), or make the “reject B” option so costly that it would never be locally rational to take it. This is stronger than Elga’s “planning”.
If the agent is capable of C:
Then, after choosing to do C and whether to accept A, there is no “choice” left to be made. The commitment just determines what will happen.
And doing C plus either accepting or rejecting A is permissible, according to any plausible choice rule applied to imprecise credences. → No sure loss.
If they aren’t capable of C:
Then I don’t see in what sense it’s irrational for the agent to have imprecise credences + locally choose whether to accept each bet based on their credences at the time of each choice.
Why? Because:
By hypothesis, they can’t force their future self to deviate from what they’d consider locally rational.
But that’s exactly what Elga’s argument tells the agent to do! [1] It says:
“Let’s grant that you and your future self might consider imprecise credences to be locally rational. [2]
“If so, your future self (choosing whether to accept B) might choose such that overall you’ve suffered a sure loss.
“You should adopt sharp credences — i.e., make your future self’s beliefs deviate from what they’d consider locally rational. That’s a strictly better alternative that you’re capable of.”
So in order for us to say the agent’s sequence of choices is irrational, we need them to be capable of a “commitment” that seems just as strong as C. [3]
(See also the links in the table here re: money pump arguments for completeness, which have a very similar structure.)
ETA (Jul 15, 2026): See here for a summary of my take on the objection: “But following C is behaviorally the same as rejecting imprecise credences (i.e., the imprecise credences don’t do any work).”
- ↩︎
H/t Jesse Clifton for making this salient to me; not sure if he’d endorse this version of the counterargument though.
- ↩︎
Indeed, Elga gives some great intuition pumps for this in the intro of his paper!
- ↩︎
You might say, adopting a different standard of rationality (as Elga asks the impreciser to do) is more psychologically tractable than C. But one way to achieve C is to adopt the principle of resolute choice.
FWIW, I don’t think Bob’s “I am uncertain about the probability” is the most plausible motivation for ranges of probabilities. (I think it’s pretty confusing for people to report ranges for this purpose, without being explicit about what the endpoints represent.)
The most plausible motivation is: “You have no reason to favor one precise probability over various others in the range. Whether you’d pick one when forced to bet is beside the point, because that doesn’t tell you what the reason is to pick one over another.” What do you think of the intuition pump in this post?
You’re not taking general claims about the epistemics of goal-directed agency and applying them to the specific case of impartially altruistic goals
I’m confused — that’s exactly what I’m doing in the argument. P2 is the general claim “there are some conditions under which we can’t compare actions’ ‘expected’ consequences”, while P3 says “for the case of impartial altruism, in our actual epistemic situation, those conditions do hold”.
It would help to hear more specific critiques of my arguments for P3, in the posts themselves (and why you find my responses to Richard weak).
Thanks!
So what if there’s a remaining degree of freedom for the prior—Savage’s theorem still holds, so even as you complain that you have no justification for one prior over another, you should still be acting as if you have one.
“Acting as if” I have a prior is compatible with admitting cluelessness. See here, which also discusses Dutch book arguments.
Solomonoff’s arguments that a simplicity prior will make only a finite number of mistakes in an approximately computable universe. You build an abstract model of how a reasoning style will perform, and then you justify using that reasoning style by appealing to good modeled performance
One problem with this is that Solomonoff priors are way too computationally intractable for us. So I don’t see what normative relevance these arguments have for us.
But when the arbitrariness is contained, it’s more appealing to say something like “This arbitrariness is unavoidable, and that’s okay. To worry that we’re making a ‘wrong choice’ or that any choice here needs a further step of justification is to misunderstand what’s going on. This is about expressing ourselves and doing our best, and it’s genuinely okay to be arbitrary in this way.”
When you say the arbitrariness is “unavoidable”, is this an implication of your previous paragraphs, or something else? I think the following response avoids some of the arbitrariness: “I simply don’t think the information I have warrants ‘expecting’ A to be better than B, or vice versa. So, I don’t have reason to c-prefer A or B. I’ll make my decisions on some basis other than c-preferences, or at the very least not tell myself I’m choosing A or B based on the impartial good, when that’s false.”
You can try to fix this by speaking of your expected value of your ideal guy’s expected value of the options, not of what you expect the guy to decide
Ah, that’s exactly what I meant — if we ourselves have precise, literal expected values about the ideal guy’s expected values. But in P1 I don’t want to assume we do. That’s why I talk about scare-quote “expectations”. I want to capture “whatever kind of aggregation across possible outcomes is EV-ish but is actually accessible to bounded agents”. (This is vague, but as I say in the footnote, it’s what EA consequentialists seem to actually appeal to in practice.)
And then, P1 says that in order for a c-preference to be justified, you need to “expect” that the literal EV you’d calculate if you were capable of doing so is positive. Does that clarify things?
Yep. Presumably BB’s implicit claim here is something like: “People make claims that we should do such and such thing because of FDT, but those claims don’t follow from ‘we should program an AI to follow FDT’.”
For example, here’s Richard Ngo saying we should cooperate with the values of civilizations outside our lightcone because of FDT.
Note that the real world contains many Newcomb-like problems. People do in fact go around making decisions that depend on their beliefs about other people’s decision algorithms.
I disagree, see here for why. (I think “decisions that depend on their beliefs about other people’s decision algorithms” is too weak to get you a Newcomblike structure.)
Thanks Vojta!
I agree that thinking about concrete scenarios is important. But I’m not exactly sure what you have in mind here: “The hope behind this is that it would give us intuition pumps with which progress on SPIs would get easier and faster.” What’s a quick example?
Is not a particularly good description of what academia and the academic peer review process is actually doing in many fields, including academic philosophy
I don’t understand why you think this. I know academia has pathologies, sure, but it seems pretty clear — as a mechanistic claim, not an “outside view heuristic” — that academic philosophers are trained to scrutinize philosophical arguments. I don’t think you’ve really explained why I should believe that “accomplished and high impact” people have more experience with scrutinizing philosophical arguments than academic philosophers.
You’re stipulating that CDT-me in your thought experiment doesn’t have access to any (psychological) actions that causally bind me to not steal from you. Right? Then sure, CDT-me would steal if he ended up in your house, and you’d want to prevent this.
But you’re also stipulating that I do have access to the action “decide to follow FDT”. That’s something that would causally bind me to not steal from you, if I took it before you made your decision whether to hire me. Why is this action a legitimate option in the hypothetical, while various other non-FDT ways of binding oneself aren’t?
Insofar as we think we should defer to some extent to [group of people] on [topic], shouldn’t we defer to [group of people whose job it is to scrutinize arguments on that topic] more so than [group of people who “are most accomplished and high impact”]? What does the latter have to do with their expertise about decision theory?
it illustrates the issue with EDT + CDT: they can’t commit to anything
What does this mean? Of course an agent who endorses EDT or CDT can commit to things — commitments are actions they decide between, like anything else.
Have you asked Katja what she’d prefer here? (I worry people might get a negative impression of her from this even though she didn’t intend it.)
It seems unnecessarily confusing to use the word “caring” for “putting weight on something in a way that isn’t ‘objective’, in the sense of empirical evidence plus logic”. (I know this has precedent in this community, tbc, I’m pushing back on that too.) I don’t assign very high probability to the sun rising tomorrow because I “care” a lot more about sun-rises-tomorrow hypotheses, I do that because I find the epistemological norms that ground induction intuitive. These are just different things, even if they share the property of being non-objective.
I believe basically everything can be formulated as a “bet”, and I don’t quite see what could be there about probabilities that can’t be phrased this way.
”What do you anticipate happening?” From my perspective, anticipation is nothing else than thinking about the consequences of an event. That’s useful if the event happens, and a waste of time if it doesn’t. Therefore, whether I anticipate an event translates to whether I want to bet my time on thinking about it.
”Aren’t you surprised by this event?” To me, surprisal is just getting into a situation that I didn’t make plans for. It’s equivalent to losing a bet: I wagered my time on thinking about the consequences of the other possibility, but the outcome that I didn’t bet on had come to pass.You can reformulate the anticipation/surprisal interpretations of probability in terms of bets, but I don’t think this is much of a positive argument for that approach. I would say, you should bet in some way for good reasons. The anticipation/surprisal interpretations at least gesture at what those reasons are: you expect better consequences from betting that way.
I think this is important, because the “probabilities are just betting odds” meme unnecessarily rules out (e.g.) imprecise probabilities by fiat.
(I’m not sure I endorse the anticipation/surprisal framings either, exactly. I think of probability more in terms of degrees of plausibility. See here for a bit more.)
good odds of changing stuff
Part of the problem OP points out is that “changing stuff” =/= “changing stuff positively”. (“Here’s one. Crucial considerations and sign flips are common.”)
What do you think of my arguments under “If they aren’t capable of C”, in the OP?