Intrinsic properties and Eliezer’s metaethics


I give an ac­count for why some prop­er­ties seem in­trin­sic while oth­ers seem ex­trin­sic. In light of this ac­count, the prop­erty of moral good­ness seems in­trin­sic in one way and ex­trin­sic in an­other. Most prop­er­ties do not suffer from this am­bi­guity. I sug­gest that this is why many peo­ple find Eliezer’s metaethics to be con­fus­ing.

Sec­tion 1: In­tu­itions of intrinsicness

What makes a par­tic­u­lar prop­erty seem more or less in­trin­sic, as op­posed to ex­trin­sic?

Con­sider the fol­low­ing three prop­er­ties that a phys­i­cal ob­ject X might have:

  1. The prop­erty of hav­ing the shape of a reg­u­lar tri­an­gu­lar. (I’ll call this prop­erty “∆-ness” or “be­ing ∆-shaped”, for short.)

  2. The prop­erty of be­ing hard, in the sense of re­sist­ing de­for­ma­tion.

  3. The prop­erty of be­ing a key that can open a par­tic­u­lar lock L (or L-open­ing-ness).

To me, in­tu­itively, ∆-ness seems en­tirely in­trin­sic, and hard­ness seems some­what less in­trin­sic, but still very in­trin­sic. How­ever, the prop­erty of open­ing a par­tic­u­lar lock seems very ex­trin­sic. (If the no­tion of “in­trin­sic” seems mean­ingless to you, please keep read­ing. I be­lieve that I ground these in­tu­itions in some­thing mean­ingful be­low.)

When I query my in­tu­ition on these ex­am­ples, it elab­o­rates as fol­lows:

(1) If an ob­ject X is ∆-shaped, then X is ∆-shaped in­de­pen­dently of any con­sid­er­a­tion of any­thing else. Ob­ject X could man­i­fest its ∆-ness even in perfect iso­la­tion, in a uni­verse that con­tained no other ob­jects. In that sense, be­ing ∆-shaped is in­trin­sic to X.

(2) If an ob­ject X is hard, then that fact does have a whiff of ex­trin­sic­ness about it. After all, X’s be­ing hard is typ­i­cally ap­par­ent only in an in­ter­ac­tion be­tween X and some other ob­ject Y, such as in a force­ful col­li­sion af­ter which the parts of X are still in nearly the same ar­range­ment.

Nonethe­less, X’s hard­ness still feels to me to be pri­mar­ily “in” X. Yes, some­thing else has to be brought onto the scene for X’s hard­ness to do any­thing. That is, X’s hard­ness can be de­tected only with the help of some “test ob­ject” Y (to bounce off of X, for ex­am­ple). Nonethe­less, the hard­ness de­tected is in­trin­sic to X. It is not, for ex­am­ple, pri­mar­ily a fact about the sys­tem con­sist­ing of X and the test ob­ject Y to­gether.

(3) Be­ing an L-open­ing key (where L is a par­tic­u­lar lock), on the other hand, feels very ex­trin­sic to me. A thought ex­per­i­ment that pumps this in­tu­ition for me is this: Imag­ine a molten blob K of metal shift­ing through a range of key-shapes. The vast ma­jor­ity of such shapes do not open L. Now sup­pose that, in the course of these meta­mor­phoses, K hap­pens to pass through a shape that does open L. Just for that in­stant, K takes on the prop­erty of L-open­ing-ness. Nonethe­less, and here is the point, an ob­server with­out de­tailed knowl­edge of L in par­tic­u­lar wouldn’t no­tice any­thing spe­cial about that in­stant.

Con­trast this with the other two prop­er­ties: An ob­server of three dots mov­ing in space might no­tice when those three dots hap­pen to fall into the con­figu­ra­tion of a reg­u­lar tri­an­gle. And an ob­server of an ob­ject pass­ing through differ­ent con­di­tions of hard­ness might no­tice when the ob­ject has be­come par­tic­u­larly hard. The ob­server can use a generic test ob­ject Y to check the hard­ness of X. The ob­server doesn’t need any­thing in par­tic­u­lar to no­tice that X has be­come hard.

But all that is just an elab­o­ra­tion of my in­tu­itions. What is re­ally go­ing on here? I think that the an­swer sheds light on how peo­ple un­der­stand Eliezer’s metaethics.

Sec­tion 2: Is good­ness in­trin­sic?

I was led to this line of think­ing while try­ing to un­der­stand why Eliezer’s metaethics is con­sis­tently con­fus­ing.

The no­tion of an L-open­ing key has been my per­sonal go-to anal­ogy for think­ing about how good­ness (of a state of af­fairs) can be ob­jec­tive, as op­posed to sub­jec­tive. The anal­ogy works like this: We are like locks, and states of af­fairs are like keys. Roughly, a state is good when it en­gages our moral sen­si­bil­ities so that, upon re­flec­tion, we fa­vor that state. Speak­ing metaphor­i­cally, a state is good just when it has the right shape to “open” us. (Here, “us” means nor­mal hu­man be­ings as we are in the ac­tual world.) Be­ing of the right shape to open a par­tic­u­lar lock is an ob­jec­tive fact about a key. Analo­gously, be­ing good is an ob­jec­tive fact about a state of af­fairs.

Ob­jec­tive in what sense? In this im­por­tant sense, at least: The prop­erty of be­ing L-open­ing picks out a par­tic­u­lar point in key-shape space1. This space con­tains a point for ev­ery pos­si­ble key-shape, even if no ex­ist­ing key has that shape. So we can say that a hy­po­thet­i­cal key is “of an L-open­ing shape” even if the key is as­sumed to ex­ist in a world that has no locks of type L. Analo­gously, a state can still be called good even if it is in a coun­ter­fac­tual world con­tain­ing no agents who share our moral sen­si­bil­ities.

But the dis­cus­sion in Sec­tion 1 made “be­ing L-open­ing” seem, while ob­jec­tive, very ex­trin­sic, and not pri­mar­ily about the key K it­self. The anal­ogy be­tween “L-open­ing-ness” and good­ness seems to work against Eliezer’s pur­poses. It sug­gests that good­ness is ex­trin­sic, rather than in­trin­sic. For, one can­not prop­erly call a key “open­ing” in gen­eral. One can only say that a key “opens this or that par­tic­u­lar lock”. But the analo­gous claim about good­ness sounds like rel­a­tivism: “There’s no ob­jec­tive fact of the mat­ter about whether a state of af­fairs is good. There’s just an ob­jec­tive fact of the mat­ter about whether it is good to you.”

This, I sup­pose, is why some peo­ple think that Eliezer’s metaethics is just warmed-over rel­a­tivism, de­spite his protes­ta­tions.

Sec­tion 3: See­ing in­trin­sic­ness in simulations

I think that we can ac­count for the in­tu­itions of in­trin­sic­ness in Sec­tion 1 by look­ing at them from the per­spec­tive simu­la­tions. More­over, this ac­count will ex­plain why some of us (in­clud­ing per­haps Eliezer) judge good­ness to be in­trin­sic.

The main idea is this: In our minds, a prop­erty P, among other things, “points to” the test for its pres­ence. In par­tic­u­lar, P evokes what­ever would be in­volved in de­tect­ing the pres­ence of P. Whether I con­sider a prop­erty P to be in­trin­sic de­pends on how I would test for the pres­ence of P — NOT, how­ever, on how I would test for P “in the real world”, but rather on how I would test for P in a simu­la­tion that I’m ob­serv­ing from the out­side.

Here is how this plays out in the cases above.

(1) In the case of be­ing ∆-shaped, con­sider a simu­la­tion (on a com­puter, or in your mind’s eye) con­sist­ing of three points con­nected by straight lines to make a tri­an­gle X float­ing in space. The points move around, and the straight lines stretch and change di­rec­tion to keep the points con­nected. The simu­la­tion it­self just keeps track of where the points and lines are. Nonethe­less, when X be­comes ∆-shaped, I no­tice this “di­rectly”, from out­side the simu­la­tion. Noth­ing else within the simu­la­tion needs to re­act to the ∆-ness. In­deed, noth­ing else needs to be there at all, aside from the points and lines. The ∆-shape de­tec­tor is in me, out­side the simu­la­tion. To make the ∆-ness of an ob­ject X man­i­fest, the simu­la­tion needs to con­tain only the ob­ject X it­self.

In sum­mary: A prop­erty will feel ex­tremely in­trin­sic to X when my de­tect­ing the prop­erty re­quires only this: “Si­mu­late just X.”

(2) For the case of hard­ness, imag­ine a com­puter simu­la­tion that mod­els mat­ter and its mo­tions as they fol­low from the laws of physics and my ex­oge­nous ma­nipu­la­tions. The simu­la­tion keeps track of only fun­da­men­tal forces, in­di­vi­d­ual molecules, and their po­si­tions and mo­menta. But I can see on the com­puter dis­play what the re­sult­ing clumps of mat­ter look like. In par­tic­u­lar, there is a clump X of mat­ter in the simu­la­tion, and I can ask my­self whether X is hard.

Now, on the one hand, I am not my­self a hard­ness de­tec­tor that can just look at X and see its hard­ness. In that sense, hard­ness is differ­ent from ∆-ness, which I can just look at and see. In this case, I need to build a hard­ness de­tec­tor. More­over, I need to build the de­tec­tor in­side the simu­la­tion. I need some other thing Y in the simu­la­tion to bounce off of X to see whether X is hard. Then I, out­side the simu­la­tion, can say, “Yup, the way Y bounced off of X in­di­cates that X is hard.” (The simu­la­tion it­self isn’t gen­er­at­ing state­ments like “X is hard”, any more than the 3-points-and-lines simu­la­tion above was gen­er­at­ing state­ments about whether the con­figu­ra­tion was a reg­u­lar tri­an­gle.)

On the other hand, cru­cially, I can de­tect hard­ness with prac­ti­cally any­thing at all in ad­di­tion to X in the simu­la­tion. I can take prac­ti­cally any old chunk of molecules and bounce it off of X with suffi­cient force.

A prop­erty of an ob­ject X still feels in­trin­sic when de­tect­ing the prop­erty re­quires only this: “Si­mu­late just X + prac­ti­cally any other ar­bi­trary thing.”

In­deed, per­haps I need only an ar­bi­trar­ily small “ep­silon” chunk of ad­di­tional stuff in­side the simu­la­tion. Given such a chunk, I can run the simu­la­tion to knock the chunk against X, per­haps from var­i­ous di­rec­tions. Then I can as­sess the re­sults to con­clude whether X is hard. The sense of in­trin­sic­ness comes, per­haps, from “tak­ing the limit as ep­silon goes to 0″, see­ing the hard­ness there the whole time, and in­ter­pret­ing this as say­ing that the hard­ness is “within” X it­self.

In sum­mary: A prop­erty will feel very in­trin­sic to X when its de­tec­tion re­quires only this: “Si­mu­late just X + ep­silon.”

(3) In this light, L-open­ing keys differ cru­cially from ∆-shaped things and from hard things.

An L-open­ing key differs from an ∆-shaped ob­ject be­cause I my­self do not en­code lock L. Whereas I can look at a reg­u­lar tri­an­gle and see its ∆-ness from out­side the simu­la­tion, I can­not do the same (let’s sup­pose) for keys of the right shape to open lock L. So I can­not simu­late a key K alone and see its L-open­ing-ness.

More­over, I can­not add some­thing merely ar­bi­trary to the simu­la­tion to check K for L-open­ing-ness. I need to build some­thing very pre­cise and com­pli­cated in­side the simu­la­tion: an in­stance of the lock L. Then I can in­sert K in the lock and ob­serve whether it opens.

I need, not just K, and not just K + ep­silon: I need to simu­late K + some­thing com­pli­cated in par­tic­u­lar.

Sec­tion 4: Back to goodness

So how does good­ness as a prop­erty fit into this story?

There is an im­por­tant sense in which good­ness is more like be­ing ∆-shaped than it is like be­ing L-open­ing. Namely, good­ness of a state of af­fairs is some­thing that I can as­sess my­self from out­side a simu­la­tion of that state. I don’t need to simu­late any­thing else to see it. Put­ting it an­other way, good­ness is like L-open­ing would be if I hap­pened my­self to en­code lock L. If that were the case, then, as soon as I saw K take on the right shape in­side the simu­la­tion, that shape could “click” with me out­side of the simu­la­tion.

That is why good­ness seems to have the same ul­ti­mate kind of in­trin­sic­ness that ∆-ness has and which be­ing L-open­ing lacks. We don’t en­code locks, but we do en­code moral­ity.


1. Or, rather, a small re­gion in key-shape space, since a lock will ac­cept keys that vary slightly in shape.