One cannot dis-identify from one’s ‘emotional side’. Thats not a thing.
In order to rationalize my emotions, I have to identify with them in the first place (as opposed to the emotions of my neighbor, say). Especially if I’m supposed to apply descriptive moral psychology, instead of just confabulating unreflectively based on whatever emotions I happen to feel at any given moment. But if I can identify with them, why can’t I dis-identify from them?
If someone thinks they’re doing that they’ve probably smuggled their emotions into their abstract reasons (see, for example, Kant).
That doesn’t stop me from trying. In fact moral psychology could be a great help in preventing such “contamination”.
You seem to be asking anti-realism to supply you with answers to normative questions. But what anti-realism tells you is that such questions don’t have factual answers.
If those questions don’t have factual answers, then I could answer them any way I want, and not be wrong. On the other hand if they do have factual answers, then I better use my abstract reasoning skills to find out what those answers are. So why shouldn’t I make realism the working assumption, if I’m even slightly uncertain that anti-realism is true? If that assumption turns out to be wrong, it doesn’t matter anyway—whatever answers I get from using that assumption, including nihilism, still can’t be wrong. (If I actually choose to make that assumption, then I must have a psychological disposition to make that assumption. So anti-realism would say that whatever normative theory I form under that assumption is my actual morality. Right?)
I’m telling you what morality is.
Can you answer the last question in the grandparent comment, which was asking just this sort of question?
If those questions don’t have factual answers, then I could answer them any way I want, and not be wrong.
That’s true as stated, but “not being wrong” isn’t the only thing you care about. According to your current morality, those questions have moral answers, and you shouldn’t answer them any way you want, because that could be evil.
When you say “you shouldn’t answer them any way you want” are you merely expressing an emotional dissatisfaction, like Jack?
If it’s meant to be more than an expression of emotional dissatisfaction, I guess “should” means “what my current morality recommends” and “evil” means “against my current morality”, but what do you mean by “current morality”? As far as I can tell, according to anti-realism, my current morality is whatever morality I have the psychological disposition to construct. So if I have the psychological disposition to construct it using my intellect alone (or any other way), how, according to anti-realism, could that be evil?
By “current morality” I mean that the current version of you may dislike some outcomes of your future moral deliberations if Omega shows them to you in advance. It’s quite possible that you have a psychological disposition to eventually construct a moral system that the current version of you will find abhorrent. For an extreme test case, imagine that your long-term “psychological dispositions” are actually coming from a random number generator; that doesn’t mean you cannot make any moral judgments today.
It’s quite possible that you have a psychological disposition to eventually construct a moral system that the current version of you will find abhorrent.
I agree it’s quite possible. Suppose I do somehow find out that the current version of me emotionally dislikes the outcomes of my future moral deliberations. I still have to figure out what to do about that. Is there a normative fact about what I should do in that case? Or is there only a psychological disposition?
I think there’s only a psychological disposition. If the future of your morals looked abhorrent enough to you, I guess you’d consider it moral to steer toward a different future.
Ultimately we seem to be arguing about the meaning of the word “morality” inside your head. Why should that concept obey any simple laws, given that it’s influenced by so many random factors inside and outside your head? Isn’t that like trying to extrapolate the eternally true meaning of the word “paperclip” based on your visual recognition algorithms, which can also crash on hostile input?
I appreciate your desire to find some math that could help answer moral questions that seem too difficult for our current morals. But I don’t see how that’s possible, because our current morals are very messy and don’t seem to have any nice invariants.
Why should that concept obey any simple laws, given that it’s influenced by so many random factors inside and outside your head?
Every concept is influenced by many random factors inside and outside my head, which does not rule out that some concepts can be simple. I’ve already given one possible way in which that concept can be simple: someone might be a strong deliberative thinker and decide to not base his morality on his emotions or other “random factors” unless he can determine that there’s a normative fact that he should do so.
Emotions are just emotions. They do not bind us, like a utility function binds an EU maximizer. We’re free to pick a morality that is not based on our emotions. If we do have a utility function, it’s one that we can’t see at this point, and I see no strong reason to conclude that it must be complex.
Isn’t that like trying to extrapolate the eternally true meaning of the word “paperclip” based on your visual recognition algorithms, which can also crash on hostile input?
How do we know it’s not more like trying to extrapolate the eternally true meaning of the word “triangle”?
But I don’t see how that’s possible, because our current morals are very messy and don’t seem to have any nice invariants.
Thinking that humans have a “current morality” seems similar to a mistake that I was on the verge of making before, of thinking that humans have a “current decision theory” and therefore we can solve the FAI decision theory problem by finding out what our current decision theory is, and determining what it says we should program the FAI with. But in actuality, we don’t have a current decision theory. Our “native” decision making mechanisms (the ones described in Luke’s tutorial) can be overridden by our intellect, and no “current decision theory” governs that part of our brains. (A CDT theorist can be convinced to give up CDT, and not just for XDT, i.e., what a CDT agent would actually self-modify into.) So we have to solve that problem with “philosophy” and I think the situation with morality may be similar, since there is no apparent “current morality” that governs our intellect.
How do we know it’s not more like trying to extrapolate the eternally true meaning of the word “triangle”?
Even without going into the complexities of human minds: do you mean triangle in formal Euclidean geometry, or triangle in the actual spacetime we’re living in? The latter concept can become arbitrarily complex as we discover new physics, and the former one is an approximation that’s simple because it was selected for simplicity (being easy to use in measuring plots of land and such). Why you expect the situation to be different for “morality”?
In order to rationalize my emotions, I have to identify with them in the first place (as opposed to the emotions of my neighbor, say). Especially if I’m supposed to apply descriptive moral psychology, instead of just confabulating unreflectively based on whatever emotions I happen to feel at any given moment. But if I can identify with them, why can’t I dis-identify from them?
I’m not sure I actually understand what you mean by “dis-identify”.
If those questions don’t have factual answers, then I could answer them any way I want, and not be wrong. On the other hand if they do have factual answers, then I better use my abstract reasoning skills to find out what those answers are. So why shouldn’t I make realism the working assumption, if I’m even slightly uncertain that anti-realism is true? If that assumption turns out to be wrong, it doesn’t matter anyway—whatever answers I get from using that assumption, including nihilism, still can’t be wrong.
So Pascal’s Wager?
In any case, while there aren’t wrong answers there are still immoral ones. There is no fact of the matter about normative ethics- but there are still hypothetical AIs that do evil things.
In any case, while there aren’t wrong answers there are still immoral ones. There is no fact of the matter about normative ethics- but there are still hypothetical AIs that do evil things.
Then there is fact of the matter about which answers are moral, and we might as well call those that aren’t, “incorrect”.
Then there is fact of the matter about which answers are moral, and we might as well call those that aren’t, “incorrect”.
It seems like a waste to overload the meaning of the word “incorrect” to also include such things as “Fuck off! That doesn’t satisfy socially oriented aspects of my preferences. I wish to enforce different norms!”
It really is useful to emphasize a carve in reality between ‘false’ and ‘evil/bad/immoral’. Humans are notoriously bad at keeping the concepts distinct in their minds and allowing ‘incorrect’ (and related words) to be used for normative claims encourages even more motivated confusion.
No. Moral properties don’t exist. What I’m doing, per the post, when I say “There are immoral answers” is expressing an emotional dissatisfaction to certain answers.
In order to rationalize my emotions, I have to identify with them in the first place (as opposed to the emotions of my neighbor, say). Especially if I’m supposed to apply descriptive moral psychology, instead of just confabulating unreflectively based on whatever emotions I happen to feel at any given moment. But if I can identify with them, why can’t I dis-identify from them?
That doesn’t stop me from trying. In fact moral psychology could be a great help in preventing such “contamination”.
If those questions don’t have factual answers, then I could answer them any way I want, and not be wrong. On the other hand if they do have factual answers, then I better use my abstract reasoning skills to find out what those answers are. So why shouldn’t I make realism the working assumption, if I’m even slightly uncertain that anti-realism is true? If that assumption turns out to be wrong, it doesn’t matter anyway—whatever answers I get from using that assumption, including nihilism, still can’t be wrong. (If I actually choose to make that assumption, then I must have a psychological disposition to make that assumption. So anti-realism would say that whatever normative theory I form under that assumption is my actual morality. Right?)
Can you answer the last question in the grandparent comment, which was asking just this sort of question?
That’s true as stated, but “not being wrong” isn’t the only thing you care about. According to your current morality, those questions have moral answers, and you shouldn’t answer them any way you want, because that could be evil.
When you say “you shouldn’t answer them any way you want” are you merely expressing an emotional dissatisfaction, like Jack?
If it’s meant to be more than an expression of emotional dissatisfaction, I guess “should” means “what my current morality recommends” and “evil” means “against my current morality”, but what do you mean by “current morality”? As far as I can tell, according to anti-realism, my current morality is whatever morality I have the psychological disposition to construct. So if I have the psychological disposition to construct it using my intellect alone (or any other way), how, according to anti-realism, could that be evil?
By “current morality” I mean that the current version of you may dislike some outcomes of your future moral deliberations if Omega shows them to you in advance. It’s quite possible that you have a psychological disposition to eventually construct a moral system that the current version of you will find abhorrent. For an extreme test case, imagine that your long-term “psychological dispositions” are actually coming from a random number generator; that doesn’t mean you cannot make any moral judgments today.
I agree it’s quite possible. Suppose I do somehow find out that the current version of me emotionally dislikes the outcomes of my future moral deliberations. I still have to figure out what to do about that. Is there a normative fact about what I should do in that case? Or is there only a psychological disposition?
I think there’s only a psychological disposition. If the future of your morals looked abhorrent enough to you, I guess you’d consider it moral to steer toward a different future.
Ultimately we seem to be arguing about the meaning of the word “morality” inside your head. Why should that concept obey any simple laws, given that it’s influenced by so many random factors inside and outside your head? Isn’t that like trying to extrapolate the eternally true meaning of the word “paperclip” based on your visual recognition algorithms, which can also crash on hostile input?
I appreciate your desire to find some math that could help answer moral questions that seem too difficult for our current morals. But I don’t see how that’s possible, because our current morals are very messy and don’t seem to have any nice invariants.
Every concept is influenced by many random factors inside and outside my head, which does not rule out that some concepts can be simple. I’ve already given one possible way in which that concept can be simple: someone might be a strong deliberative thinker and decide to not base his morality on his emotions or other “random factors” unless he can determine that there’s a normative fact that he should do so.
Emotions are just emotions. They do not bind us, like a utility function binds an EU maximizer. We’re free to pick a morality that is not based on our emotions. If we do have a utility function, it’s one that we can’t see at this point, and I see no strong reason to conclude that it must be complex.
How do we know it’s not more like trying to extrapolate the eternally true meaning of the word “triangle”?
Thinking that humans have a “current morality” seems similar to a mistake that I was on the verge of making before, of thinking that humans have a “current decision theory” and therefore we can solve the FAI decision theory problem by finding out what our current decision theory is, and determining what it says we should program the FAI with. But in actuality, we don’t have a current decision theory. Our “native” decision making mechanisms (the ones described in Luke’s tutorial) can be overridden by our intellect, and no “current decision theory” governs that part of our brains. (A CDT theorist can be convinced to give up CDT, and not just for XDT, i.e., what a CDT agent would actually self-modify into.) So we have to solve that problem with “philosophy” and I think the situation with morality may be similar, since there is no apparent “current morality” that governs our intellect.
Even without going into the complexities of human minds: do you mean triangle in formal Euclidean geometry, or triangle in the actual spacetime we’re living in? The latter concept can become arbitrarily complex as we discover new physics, and the former one is an approximation that’s simple because it was selected for simplicity (being easy to use in measuring plots of land and such). Why you expect the situation to be different for “morality”?
I’m not sure I actually understand what you mean by “dis-identify”.
So Pascal’s Wager?
In any case, while there aren’t wrong answers there are still immoral ones. There is no fact of the matter about normative ethics- but there are still hypothetical AIs that do evil things.
Which question exactly?
Then there is fact of the matter about which answers are moral, and we might as well call those that aren’t, “incorrect”.
It seems like a waste to overload the meaning of the word “incorrect” to also include such things as “Fuck off! That doesn’t satisfy socially oriented aspects of my preferences. I wish to enforce different norms!”
It really is useful to emphasize a carve in reality between ‘false’ and ‘evil/bad/immoral’. Humans are notoriously bad at keeping the concepts distinct in their minds and allowing ‘incorrect’ (and related words) to be used for normative claims encourages even more motivated confusion.
No. Moral properties don’t exist. What I’m doing, per the post, when I say “There are immoral answers” is expressing an emotional dissatisfaction to certain answers.
True.