They’d re-define their policies to protect or multiply whatever objects of value actually do exist in the world, or attempt to create the valuable things that turned out not to exist (e. g., the kid baking a cake on his own).
Well, yes, assuming there still are other things of value for them, at least potentially. But if a robot had been a hedonistic consequentialist, then learned that itself and all beings in its universe lacked any phenomenal experiences (and that such experiences were impossible in its universe), that robot wouldn’t have any further set of objects to value while still being a hedonistic consequentialist.
You seem to be saying: well, when people say “all I really value is X” they don’t really mean that literally. But sometimes, for some agents, it is true that all they value is X. In such cases, there is no further set of things to still value. Utilitarians might fit this description of such an agent. This is why NickGabs had mentioned the hard problem of consciousness.
Okay, yes. One interpretation of the consequentialism I mentioned degrades gracefully like you say, giving “all zeroes” to a world of mere automata. (That’s a point in favor of such a consequentialism that it cannot ever become internally inconsistent.) But what you’re saying in the general case doesn’t fit what people usually mean when they talk about their values. A world without a God is not merely a world with lower utility for a theist, because the concept of “world” is already defined in terms of a God for the theist to begin with. You think otherwise perhaps because you have carved the concept-space up in such a way that lets you take apart different ideas like “world”, “God”, “value” and evaluate them separately. But this is not the case for the theist. Otherwise, they would not be a theist.
You’re right that it ought to work that way, though.
… And so when such a theist discovers that their world-model is fundamentally wrong, they will re-carve it in a way such that they can take “God” out of it, and then find things they can value in the world that actually exists, or, failing that, would assign it zero value and completely break down. No?
I mean, what are you suggesting? What do you think a theist who discovers that God doesn’t exist would do, that wouldn’t fit into one of the three scenarios I’d outlined above?
A theist who has apostatized may still be the same person afterwards, but by no means are they the same agency. One agent has simply died, being replaced with a new one.
Either there are some concrete things corresponding to personal identity/preference satisfaction/etc., or we value not these things but some actually-existing correlates of these things, or acting like we value these things is a heuristic that instrumentally helps us arrive at good outcomes. Either way, ontology shifts don’t do anything bad to our values.
Leading to this claim, which is very alluring to me, even if I disagree with it:
Or, the other way around, perhaps “values” are defined by being robust to ontology shifts.
You gave the example that us learning that our world is physical didn’t make us value humans less, even if we no longer believe that they have immortal souls. That’s true, we still value humans. But the problem is that the first group of agents (medieval “we”) are a different group than the second group (post-industrial “we”). Each group has a different concept of “human”. The two concepts roughly map to the same targets in the territory, but the semantic meaning is different in a way that is crucial to the first group of agents. The first group weren’t convinced that they were wrong. They simply got replaced by a new group who inherited their title.
That might sound dramatic, but ask a truly committed religious person if they can imagine themselves not believing in God. They will not be able to imagine it. This is because, in fact, they would no longer be the same agency in the sense that is crucial to them. Society would legally consider them the same person, and their apostate future-self might claim the same title as them and even deny that the change really mattered, but the original entity who was asked, the theistic agent, would no longer recognize any heir as legitimate at that point. From its point of view, it has simply died. That is why the theist cannot even imagine becoming an apostate, even if they grant that there’s a greater than zero chance of becoming an apostate at all times.
It’s true what you say, if you’re talking about people i.e. biological human organisms. Such entities will always be doing something in the world up until the very moment they physically expire, including having their whole world view shattered and living in the aftermath of that. However, pre-world-view-shattering them would not recognize post-world-view-shattering them as a legitimate heir. The person might be the same, but it’s a different agency in control.
Similar things can be said about populations and governments.
I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.
Well, yes, assuming there still are other things of value for them, at least potentially. But if a robot had been a hedonistic consequentialist, then learned that itself and all beings in its universe lacked any phenomenal experiences (and that such experiences were impossible in its universe), that robot wouldn’t have any further set of objects to value while still being a hedonistic consequentialist.
You seem to be saying: well, when people say “all I really value is X” they don’t really mean that literally. But sometimes, for some agents, it is true that all they value is X. In such cases, there is no further set of things to still value. Utilitarians might fit this description of such an agent. This is why NickGabs had mentioned the hard problem of consciousness.
Sure. So the universe is worthless for such agents. I don’t see how that’s a problem with my model...?
Okay, yes. One interpretation of the consequentialism I mentioned degrades gracefully like you say, giving “all zeroes” to a world of mere automata. (That’s a point in favor of such a consequentialism that it cannot ever become internally inconsistent.) But what you’re saying in the general case doesn’t fit what people usually mean when they talk about their values. A world without a God is not merely a world with lower utility for a theist, because the concept of “world” is already defined in terms of a God for the theist to begin with. You think otherwise perhaps because you have carved the concept-space up in such a way that lets you take apart different ideas like “world”, “God”, “value” and evaluate them separately. But this is not the case for the theist. Otherwise, they would not be a theist.
You’re right that it ought to work that way, though.
… And so when such a theist discovers that their world-model is fundamentally wrong, they will re-carve it in a way such that they can take “God” out of it, and then find things they can value in the world that actually exists, or, failing that, would assign it zero value and completely break down. No?
I mean, what are you suggesting? What do you think a theist who discovers that God doesn’t exist would do, that wouldn’t fit into one of the three scenarios I’d outlined above?
A theist who has apostatized may still be the same person afterwards, but by no means are they the same agency. One agent has simply died, being replaced with a new one.
I’d be genuinely interested if you could elaborate on that, including the agency vs. personality distinction you’re making here.
Okay. Let me back up a bit. You had said:
Leading to this claim, which is very alluring to me, even if I disagree with it:
You gave the example that us learning that our world is physical didn’t make us value humans less, even if we no longer believe that they have immortal souls. That’s true, we still value humans. But the problem is that the first group of agents (medieval “we”) are a different group than the second group (post-industrial “we”). Each group has a different concept of “human”. The two concepts roughly map to the same targets in the territory, but the semantic meaning is different in a way that is crucial to the first group of agents. The first group weren’t convinced that they were wrong. They simply got replaced by a new group who inherited their title.
That might sound dramatic, but ask a truly committed religious person if they can imagine themselves not believing in God. They will not be able to imagine it. This is because, in fact, they would no longer be the same agency in the sense that is crucial to them. Society would legally consider them the same person, and their apostate future-self might claim the same title as them and even deny that the change really mattered, but the original entity who was asked, the theistic agent, would no longer recognize any heir as legitimate at that point. From its point of view, it has simply died. That is why the theist cannot even imagine becoming an apostate, even if they grant that there’s a greater than zero chance of becoming an apostate at all times.
It’s true what you say, if you’re talking about people i.e. biological human organisms. Such entities will always be doing something in the world up until the very moment they physically expire, including having their whole world view shattered and living in the aftermath of that. However, pre-world-view-shattering them would not recognize post-world-view-shattering them as a legitimate heir. The person might be the same, but it’s a different agency in control.
Similar things can be said about populations and governments.
Thanks for elaborating!
I think I get what you mean. Human minds’ “breakdown protocol” in case the universe turns out to be empty of value isn’t to just shut down; the meat keeps functioning, so what happens is the gradual re-assembly of a new mind/agent from the pieces of the old one. Does that pass the ITT?
But I remain unconvinced that this is what happens during a crisis of faith. By themselves, the theist’s refusal to admit their future apostate self as an heir, and their belief that they’d die if they were to lose faith, don’t mean much to me beyond “the religion memeplex encourages its hosts to develop such beliefs to strengthen its hold on them”. Especially if the apostate later denies that these beliefs were true.
And while my model of a devout mind in the middle of a crisis of faith is indeed dramatic and involves some extreme mental states… I’m unconvinced that this involves changes to terminal values. The entire world-model and the entire suite of instrumental values being rewritten is dramatic enough on its own, and “terminal values” slot nicely into the slot of “what’s guiding this rewriting”/”what’s the main predictor of what the apostate’s instrumental values will be”.
Building off of your other answer here, I think I can imagine at least one situation where your terminal values will get vetoed. Imagine that you discovered, to your horror, that all of your actions up until now have been subconsciously motivated to bring about doomsday. Causing the death of everyone is actually your terminal goal, which you were ignorant of. Furthermore, your subconsciously motivated actions actually have been effective at bringing the world closer and closer to its demise. Your only way to divert this now is to throw yourself immediately out of the window to your death, thereby averting your own terminal goal.
Would you do this?
Dr. Jekyll and Mr. Hyde are the same person, but are they really the same agent?
Suppose I would end up walking out the window. And it would be the wrong action for me to take. I would be foiled by a bunch of bad heuristics and biases I’d internalized over the course of my omnicidal plot. There would be no agent corresponding to me whose values would be satisfied by this.
It would be not unlike, say, manipulating and gaslighting someone until they decide to kill their entire family. This would be against the values the person would claim as their “truer” ones, but in the moment, under the psychological pressure and the influence of some convincing lies, it’d (incorrectly) feel to them like a good idea.