Okay, thanks. Yeah, to me mundane correlations are far more important than acausal trade (although I haven’t thought about acausal trade enough to be confident).
I feel that “just do what seems best based on causal considerations” seems almost always the right choice. I think maybe the main deviation from the causally optimal strategy is some amount of extra kindness towards beings with different values. But I think this effect is pretty weak in practice.
Okay, I see—this feels quite central to me since the whole framework depends on this, right?
As I say in my edit in the top-level comment, my understanding is that OSAC is the caring measure approach made somewhat less arbitrary by trying to derive plausible conclusions from plausible high-level moral preferences—so that you can’t just say “Oh, but I care about this more” to justify literally any random decision (which might be seen as a reductio, c.f. Yudkowsky’s lottery thing).
So the reasoning of “conditioning on observing a simple world, having conventionally reasonable beliefs (e.g. that you won’t win the lottery) is the best strategy” doesn’t work otherwise.
But I do agree with you, actually—if you have a moral parliament, this does seem like it works, since my action’s effect seems like it should be a much larger fraction of utility of the short-description-length moments than of e.g. all Boltzmann brains. So the mathematically simple worlds bid extremely strongly. That’s pretty cool. I think I get it now.
It wasn’t clear to me how load-bearing the moral parliament / bounded utilities aspect is until now.
Just to check, do you agree that if someone doesn’t have a moral parliament (e.g. cares only about the impartial good, or has moral uncertainty but just aggregates), this whole thing blows up and you cannot do this “systematization of the caring measure” thing? Since e.g. even small weight on each Boltzmann brain will blow up? So for example, caring measure + classic hedonistic utilitarianism does not allow you to recover normal causal reasoning (at least in this way)?
(This isn’t a rhetorical question, by the way—it’s been a real update for me from thinking about this kind of stuff, cooperation and not painfully forcing myself to do stuff (because of the predominance of freak observers) are more important to me now—I’m guessing more important than they are to you. So I’ve been confused until now how you come to a different conclusion, but now I get that it’s because of the moral parliament thing).
Yes, the bounded utilities aspect is very crucial. I think it’s important that if you have two importantly different theories/distributions/moral preferences/agents in your moral parliament, and one of them suddenly comes up with a new calculation that says there are a trillion times more affected people under their theory than previously thought, then they shouldn’t be able to entirely overwhelm the other theory. They should get some advantage—I put some weight on a distribution under theory A, a distribution under theory B, and a combined distribution under theory A and B. If theory A realizes that there are a trillion times more people in it than expected, then it’s allowed to overwhelm the combined distribution, but it’s not allowed to touch the distribution allocated to theory B, so theory B still has some weight.
As for the moral parliament imagery: I think it’s a good picture, and I should have probably used it more in this post. The reason I didn’t use it as that usually moral parliament invokes the image of negotiations, non-VNM systems, geometric rationality, etc. While I couldn’t really find a thought experiment yet where I felt that was needed, and so for now OSAC is equivalent to the picture of a bunch of people sitting in a moral parliament, all having a bounded utility function going up to 1, but caring about different things, and I always try to make the decision that maximizes the sum of utilities in the parliament. So it’s a simpler picture than how moral parliaments are often conceived. (I find it somewhat likely that I will later find a thought experiment were this summation fails, and then I need to think of more complicated decision processes.)
After some thought, I do agree that a parliament is important at the very least in practice, and probably in theory as well. And that this does lead to a large bid for normal causal reasoning.
But I think there are ways to make this compatible with my intuition that freak observers and cooperation matter a lot for acausal reasons—e.g. if I have a faction in my moral parliament that cares about my freak observer selves (not about freak observers in general), or if I have one that cares about the part of the global multiversal hedonic utility that I specifically can affect (i.e. the small fraction that I can affect by cooperating more). Those are pretty natural IMO, and would also bid very strongly for my current actions. But I guess they’re more indexed on myself / selfish than anything you’re doing? Unsure.
Or maybe this is better: I could actually just have a parliament for theories about the world or broad worldviews, since that also avoids fanaticism (this is more geometric rationality flavored, I think?). And my “moral value” is just doing the impartial good / right thing in some vague way. This feels more natural anyway since moral values are deeply bound up in worldviews and perceptions of what the world is like.
Then, if I gain a lot of confidence in the worldview of a multiverse, it can bid with decent strength for a lot of broad cooperation and not forcing things.
(the difference with the moral parliament picture is that it feels more natural that a holistic worldview can bid with some strength even if I am small relative to the world in that worldview (because holistic worldviews don’t generally tell you that you don’t matter, i.e. there’s a sort of minimum meaningfulness of your actions). Unlike a utility function over the multiverse, which might not care about my actions at all).
So this would avoid standard fanaticism (Pascal’s mugging) while still being able to appreciate the predominance of freak observers. I kinda like this.
Okay, thanks. Yeah, to me mundane correlations are far more important than acausal trade (although I haven’t thought about acausal trade enough to be confident).
Okay, I see—this feels quite central to me since the whole framework depends on this, right?
As I say in my edit in the top-level comment, my understanding is that OSAC is the caring measure approach made somewhat less arbitrary by trying to derive plausible conclusions from plausible high-level moral preferences—so that you can’t just say “Oh, but I care about this more” to justify literally any random decision (which might be seen as a reductio, c.f. Yudkowsky’s lottery thing).
So the reasoning of “conditioning on observing a simple world, having conventionally reasonable beliefs (e.g. that you won’t win the lottery) is the best strategy” doesn’t work otherwise.
But I do agree with you, actually—if you have a moral parliament, this does seem like it works, since my action’s effect seems like it should be a much larger fraction of utility of the short-description-length moments than of e.g. all Boltzmann brains. So the mathematically simple worlds bid extremely strongly. That’s pretty cool. I think I get it now.
It wasn’t clear to me how load-bearing the moral parliament / bounded utilities aspect is until now.
Just to check, do you agree that if someone doesn’t have a moral parliament (e.g. cares only about the impartial good, or has moral uncertainty but just aggregates), this whole thing blows up and you cannot do this “systematization of the caring measure” thing? Since e.g. even small weight on each Boltzmann brain will blow up? So for example, caring measure + classic hedonistic utilitarianism does not allow you to recover normal causal reasoning (at least in this way)?
(This isn’t a rhetorical question, by the way—it’s been a real update for me from thinking about this kind of stuff, cooperation and not painfully forcing myself to do stuff (because of the predominance of freak observers) are more important to me now—I’m guessing more important than they are to you. So I’ve been confused until now how you come to a different conclusion, but now I get that it’s because of the moral parliament thing).
Yes, the bounded utilities aspect is very crucial. I think it’s important that if you have two importantly different theories/distributions/moral preferences/agents in your moral parliament, and one of them suddenly comes up with a new calculation that says there are a trillion times more affected people under their theory than previously thought, then they shouldn’t be able to entirely overwhelm the other theory. They should get some advantage—I put some weight on a distribution under theory A, a distribution under theory B, and a combined distribution under theory A and B. If theory A realizes that there are a trillion times more people in it than expected, then it’s allowed to overwhelm the combined distribution, but it’s not allowed to touch the distribution allocated to theory B, so theory B still has some weight.
As for the moral parliament imagery: I think it’s a good picture, and I should have probably used it more in this post. The reason I didn’t use it as that usually moral parliament invokes the image of negotiations, non-VNM systems, geometric rationality, etc. While I couldn’t really find a thought experiment yet where I felt that was needed, and so for now OSAC is equivalent to the picture of a bunch of people sitting in a moral parliament, all having a bounded utility function going up to 1, but caring about different things, and I always try to make the decision that maximizes the sum of utilities in the parliament. So it’s a simpler picture than how moral parliaments are often conceived. (I find it somewhat likely that I will later find a thought experiment were this summation fails, and then I need to think of more complicated decision processes.)
Cool, gotcha.
After some thought, I do agree that a parliament is important at the very least in practice, and probably in theory as well. And that this does lead to a large bid for normal causal reasoning.
But I think there are ways to make this compatible with my intuition that freak observers and cooperation matter a lot for acausal reasons—e.g. if I have a faction in my moral parliament that cares about my freak observer selves (not about freak observers in general), or if I have one that cares about the part of the global multiversal hedonic utility that I specifically can affect (i.e. the small fraction that I can affect by cooperating more). Those are pretty natural IMO, and would also bid very strongly for my current actions. But I guess they’re more indexed on myself / selfish than anything you’re doing? Unsure.Or maybe this is better: I could actually just have a parliament for theories about the world or broad worldviews, since that also avoids fanaticism (this is more geometric rationality flavored, I think?). And my “moral value” is just doing the impartial good / right thing in some vague way. This feels more natural anyway since moral values are deeply bound up in worldviews and perceptions of what the world is like.
Then, if I gain a lot of confidence in the worldview of a multiverse, it can bid with decent strength for a lot of broad cooperation and not forcing things.
(the difference with the moral parliament picture is that it feels more natural that a holistic worldview can bid with some strength even if I am small relative to the world in that worldview (because holistic worldviews don’t generally tell you that you don’t matter, i.e. there’s a sort of minimum meaningfulness of your actions). Unlike a utility function over the multiverse, which might not care about my actions at all).
So this would avoid standard fanaticism (Pascal’s mugging) while still being able to appreciate the predominance of freak observers. I kinda like this.