You can try to fix this by speaking of your expected value of your ideal guy’s expected value of the options, not of what you expect the guy to decide
Ah, that’s exactly what I meant — if we ourselves have precise, literal expected values about the ideal guy’s expected values. But in P1 I don’t want to assume we do. That’s why I talk about scare-quote “expectations”. I want to capture “whatever kind of aggregation across possible outcomes is EV-ish but is actually accessible to bounded agents”. (This is vague, but as I say in the footnote, it’s what EA consequentialists seem to actually appeal to in practice.)
And then, P1 says that in order for a c-preference to be justified, you need to “expect” that the literal EV you’d calculate if you were capable of doing so is positive. Does that clarify things?
Ah, that’s exactly what I meant — if we ourselves have precise, literal expected values about the ideal guy’s expected values. But in P1 I don’t want to assume we do. That’s why I talk about scare-quote “expectations”. I want to capture “whatever kind of aggregation across possible outcomes is EV-ish but is actually accessible to bounded agents”. (This is vague, but as I say in the footnote, it’s what EA consequentialists seem to actually appeal to in practice.)
And then, P1 says that in order for a c-preference to be justified, you need to “expect” that the literal EV you’d calculate if you were capable of doing so is positive. Does that clarify things?