Eliezer says that when he uses words like “moral”, “right”, and “should”, he’s referring to properties of a specific computation. That computation is essentially an idealized version of himself (e.g., with additional resources and safeguards).
When we use words like “right”, we don’t refer to “right”, we refer to particular heuristics that allow us to make right decisions. These heuristics are indeed properties of us, and improved heuristics are properties of idealized versions of us. But “right” itself is not a property of humans, or the kind of thing we can build or imagine. It’s a different kind of referent from the usual concepts, for some reason. We don’t refer to “right”, we use it through particular heuristics.
Consider an analogy with mathematical truth. When we use “truth”, do we refer to particular heuristics that allow us to know which mathematical statements are true, or do we refer to something which exists largely independently of human beings, even though we’re not really sure what it is exactly? The latter makes more sense to me.
I do not mean to imply that by “right” we must also refer to something that exists independently of human beings, but I think this analogy shows that we need more than “we can’t seem to figure out what ‘right’ refers to, but clearly these heuristics have something to do with it” in order to conclude that the heuristics are what we mean by “right”.
“we can’t seem to figure out what ‘right’ might refer to besides these heuristics”
“Right” as the meta-ethical question doesn’t refer to heuristics, but when we talk of “right”, what we often mean to do is to refer to those heuristics (but not always). So I didn’t contrast the two senses you discuss in the first paragraph.
When one speaks, in some sense one always argues. So when you say that “saving that child was right”, you appeal to particular moral intuitions, i.e. heuristics, allowing other people to notice these intuitions and agree with you, not making a meta-ethical claim about this action being in accordance with the essence of “right”.
When we use words like “right”, we don’t refer to “right”, we refer to particular heuristics that allow us to make right decisions. These heuristics are indeed properties of us, and improved heuristics are properties of idealized versions of us. But “right” itself is not a property of humans, or the kind of thing we can build or imagine. It’s a different kind of referent from the usual concepts, for some reason. We don’t refer to “right”, we use it through particular heuristics.
Consider an analogy with mathematical truth. When we use “truth”, do we refer to particular heuristics that allow us to know which mathematical statements are true, or do we refer to something which exists largely independently of human beings, even though we’re not really sure what it is exactly? The latter makes more sense to me.
I do not mean to imply that by “right” we must also refer to something that exists independently of human beings, but I think this analogy shows that we need more than “we can’t seem to figure out what ‘right’ refers to, but clearly these heuristics have something to do with it” in order to conclude that the heuristics are what we mean by “right”.
“Right” as the meta-ethical question doesn’t refer to heuristics, but when we talk of “right”, what we often mean to do is to refer to those heuristics (but not always). So I didn’t contrast the two senses you discuss in the first paragraph.
Do we really often mean to refer to those heuristics by “right”? Can you give me a couple of examples of when we clearly intend to do that?
When one speaks, in some sense one always argues. So when you say that “saving that child was right”, you appeal to particular moral intuitions, i.e. heuristics, allowing other people to notice these intuitions and agree with you, not making a meta-ethical claim about this action being in accordance with the essence of “right”.
Sometimes when one speaks one speaks for oneself!