I’m interested in axiology/value/utility—what things are valuable and why and when and how and to whom?
ihatenumbersinusernames7
Karma: 74
don’t overlook how cheesy gimmicks can help a cause (the Trump-as-a-McDonalds-worker photo is a great example of this). I suggest someone buys Eliezer a Ford F-150 and a cowboy hat, and organizes a road trip where he drives around to remote rural towns to spread the word of AI safety.
I chuckled, then thought for a few seconds, and that’s actually not so bad an idea.
Can I get the best bot I can for which Talker is plausibly in control?
I think what you describe here:
A Doer part that manages a Talker part to help steer reward where it needs to go and satiate the user’s questions, instead of a Talker that often initiates Doer stuff to succeed at the present conversation.
...is not an emergence a Fable 5 or Sol 5.6. (I’ve read through a couple times and can’t figure out what Eliezer thinks is different from earlier models.)
Or instead of Talker being directly subordinate to Doer, maybe Germany/Berlin <> Doer, but rather Germany/Berlin = [command center / J Space / whatever] that runs both Doer and Talker.
So if Talker was (ever) plausibly in control, I think that was an illusion.
could’ve done more to save the world by helping the public push on AI labs from outside instead.
Citation needed? How? I guess I just think, thanks for trying, however you do it. And it’s good that we have people trying both inside and out.
the more I felt that wanting ethical realism to be true was no different from wanting religion to be true.
Interesting...do you want it to be true? By my lights that’s not the same as having the intuition that it is true.
I still feel like 0 intuition for ethical realism: I can have strong feelings about some stuff, but I can easily explain them away as a mere result of historically fit natural-and-social evolutionarily useful strategies that get coded in our genes and...
As we’ve touched on, reason can also be explained away as an evolutionary useful, gene encoded, quirk. But as you’ve said before, reason seems to reflect the way the world is. I think ethical intuitions do too.
So these kinds of debates do feel rather circular if they just end up with ’okay, I have these intuitions and you have yours.
The “strong feelings about some stuff”—the particular beliefs—aren’t the intuitions I’m defending. The intuition I think we share is that we have the strong feelings at all. We can explain it away, but it does seem to be a fact about how humans are. That doesn’t make it objective (tier 1) but maybe it’s intersubjective (tier 2). It seems like we might even agree on that...which means whether or not we call that moral realism is just arguing about words. And maybe that’s why Yud opts for the object level here—because the meta level is where we’re tempted to say, “If a person is murdered in cold blood, most everyone agrees that’s bad, but is it...objectively bad?”
And when I ask or look for clarifications of why a realist holds the belief, I just get extremely insatisfactory answers, almost all of which are variants of ’well, these are self-evident intuitions that I have”.
Delete all the intuitions, and you aren’t left with an ideal philosopher of perfect emptiness, you’re left with a rock.
Keep all your specific intuitions and refuse to build upon the reflective ones, and you aren’t left with an ideal philosopher of perfect spontaneity and genuineness, you’re left with a grunting caveperson running in circles, due to cyclical preferences and similar inconsistencies.
“Intuition”, as a term of art, is not a curse word when it comes to morality—there is nothing else to argue from. Even modus ponens is an “intuition” in this sense—it’s just that modus ponens still seems like a good idea after being formalized, reflected on, extrapolated out to see if it has sensible consequences, etcetera.
https://www.lesswrong.com/posts/r5MSQ83gtbjWRBDWJ/the-intuitions-behind-utilitarianism
I agree with Yudkowsky on this. The consequences of ignoring our intutitions aren’t much better than the consequences of not questioning them at all.
whereas in tier 2, the map (also human beliefs and interpretations) changes but the territory (=moral truths?) might not?
Yep, that’s what I was saying. If moral realism is correct, that’s how it works. (I think)
like most people, I recoil at violence being exerted against the helpless. But this recoiling and the emotional effects are something I can easily not take serious as truth-describing, in just the same way as I can easily not take as ‘truth-describing’ that fatty, sugary foods are good for me, even if my brain rewards me for eating them. I’d say my base rate for scenarios like the one you depict is something like this:
Well, your point was about preferring vanilla to other flavors, not about believing ice cream was good for you. Preferences are tier 2, the facts of “good for me” (depending how we define that) seem potentially tier 1.
But anyway, I think I get what you’re saying about preferences—we don’t really have a way to say what they “should” be, if indeed what they are is just a mess of biases and whatever we got from evolution etc. My point was that if moral realism is correct, they’re not that. They’re some sort of best-guess instrumental values we have derived from our terminal value of “the good.” But I admit I don’t have a testable experiment that could prove this...just the intuition that some actions we take have qualities and consequences that make reality more “good” or less “good.”
Saying, “stealing is wrong, but I would really like to have X” might actually help you achieve your goal.
To argue back at myself, I guess what the moral realist would say is, “Your goal is wrong, and unaligned with the correct goal of [whatever is morally true]. You need to update the instrumental value(s) that don’t follow from your/the true (unbeknownst to you) terminal value.”
there are no should with tier one either: there’s just truths about an external world that are independent of human desires and cognition. You can ignore or reject them, but reality will reassert itself in spite of your wishes, or those of the whole human community. You can ignore reality, at your peril.
The territory doesn’t change, agreed.
For tier 2, individuals seldom can go unscathed by ignoring or rejecting the societal shoulds, but they change with times and with movements in favor of changes in some directions.
I’m saying the map changes but the territory might not. When many thought slavery ok, that world was in some real way worse for it. Even though the norms did not reflect that at the time.
My claim is, with respect to shoulds, everything is of the same reference class as ‘I prefer vanilla ice-cream’ or ‘I prefer blue’.
This is the crux. Most of what I’ve been saying is provisional...I’m not certain. But this statement strikes me as utterly counterintuitive. I know this is fairly standard non-cognitivist claim...I just can’t imagine believing it practically. If someone attacks me physically, unprovoked, as I’m walking down the street, I don’t think “a norm has been broken” or “my preference is that you don’t do that.” I guess to be fair, I also don’t immediately think “your instrumental values are unaligned with your true terminal values,” haha. But I think I really believe that.
There’s like three tiers here: stuff that’s mind-independent (it doesn’t matter what any mind or group of minds believe and think with respect to the underlying truth), stuff that’s intersubjective (which is mind-dependent but not amenable to change by an individual) and purely subjective stuff (which is completely dependent on the subject’s subjective beliefs, feelings, whatever).
You put ‘agreed social norms’ in the 2nd tier, but I don’t think it’s just consensus that makes this category meaningful. If it’s real, there has to be a correct and incorrect to it. I don’t think slavery is wrong because of consensus, and that it could be right if (or when) most people agree it is.
Consensus is just a form of evidence that “intersubjectivity” [we can use that as a stand in for value realism] is a real thing. The Mona Lisa is better than a stick figure. “The Long and Winding Road” is a better song than “Mary had a little lamb.” I don’t think the reason it’s better is because most people agree it is. I think it’s because of real human preferences/values we have...but that we can ignore or just be foolish or wrong about. It’s a subtle difference, but I think it’s everything.
We don’t need consensus on everything. The fact that we do have wide consensus on so many things, and the fact that we feel we should converge on things (so we argue)...these are still evidence.
So I think that maybe there is a truth about what some (probably not all*) values/preferences should be. If you could produce a person that prefers “Mary had a little lamb,” maybe we could say that person is objectively wrong about intersubjective value. But they can still be right about their own (bad) opinion.
*I don’t think this means it’s right to prefer blue over green. And no I don’t have any idea how to determine which values are real (intersubjective) and which are just opinion.
I just don’t think it is doing any truth-tracking about reality.
Well, it tracks the truth that human values do exist, even if they vary somewhat. But agreed that it doesn’t tell you that there’s a source for normative values, or what that source might be.
Reason can tell you that your beliefs are inconsistent, or that your chosen means won’t achieve your ends, but I don’t see how reason by itself can supply the terminal ends.
Such a good point. Instrumental values are in the realm of logic (logically derived from terminal values) -- so given a terminal value, we can be logically right or wrong about the instrumental values that follow.
I have wondered (but perhaps haven’t stated clearly) if it is a fact about conscious minds that we have but one terminal value—a vague desire for “the good.” All our other values, even health, love, and not-misery, are instrumentally derived. If it is a fact that minds desire what they deem good (not that I can prove it), would that sway you at all toward moral realism?
In other words if it is a fact about minds that they desire the good, then the question becomes not “does goodness exist” but “what is good?”
Or would you just say well that makes goodness real for minds, but not objectively? This may be fair, but what does “objectively” mean in this case?
I agree, with the one caveat that I think there is a detectable difference in a moral world and an amoral world. But the difference is normative and not absolute. If someone thinks misery is just as good as flourishing, then there’s no way to differentiate.
...but what about someone who doesn’t believe in non-contradtiction? He is logically confused; the misery-preferer is normatively confused.
When you say “2 + 2 = 4” and then find 2 of something and 2 more and count 4, a logical intuition is instantiated.
When you say, “It would be good to help that old lady across the street,” and then you don’t help her, and regret it, a moral intuition is instantiated.
...I think?
Agreed that empirical knowledge can be tested, but what about knowledge not gained through the senses? You believe in the law of non-contradiction, right? There’s no experiment that can be run to prove it’s true. Maybe logical beliefs are similar to moral beliefs?
“Since a reward function external to AlphaZero, that “value” doesn’t change the way ours do since our values are within us.”
I still wonder if AlphaZero’s instrumental values can change...but I guess, since the reward function is fixed and external, optimizing based that reward keeps the instrumental values from getting weird.
Or here’s a better intuition pump: argue for something you believe in vehemently, but don’t act like it’s actually true.
Re: moral realism, I’ve got 2 for you:
Think about whatever you believe is right (or wrong) that you would argue for. You probably don’t argue that it’s just your opinion.
We can’t prove modus ponens or that A=A. We can’t prove that we’re not in a simulation. Therefore math and induction and science are all theories. We can’t have certainty. But we have intuitions about logic as well as morality.
By contrast, if you are in the endgame of a game of chess, or in an oligopolistic competition, you absolutely need to think about how other players will respond to your actions. You do need to search through a tree of “if I do this, that will happen” and compute that specifically for the specific game state you’re in, and recompute it every time the game state changes.
Since chess is bounded you really can calculate (and in the endgame you’re now able to). With oligopolistic competition, your power has increased and the players that matter to you are fewer, but the moves are not bounded—if Apple and Dell (or whoever) are in an oligopoly but then Apple creates the iPhone...things can change.
Another helpful way of putting #2 is: “Given that she can only win once”
As many have pointed out in the past, if this is the case, in the tails scenario she always wins on Mon(tails), so Tues(tails) never actually happens, because the game is over.
It will remain unsolved as long as reasonable people continue to disagree.
I’m not sure people disagree:
Given [her waking up], not knowing what day it is, she’s in the tails world 2⁄3 of the time.
Given [we’re about to play this game], there’s a 50⁄50 chance of heads or tails. So whatever day she wakes up on gives you no further information.
Does anyone disagree?
Yes! And I think it’s encouraging that some are trying.