rot13
Jnf lbhe nafjre fbzrguvat nybat gur yvarf bs “V jbefuvc Tbq be V nz zneevrq, ohg abg obgu”?
AprilSR
Karma: 146
I feel like there are two obvious strategies here:
Make a lot of money, somehow
Break cryptography
I can’t think of anything other than those which would be useful without requiring you give the AI lots of information about the physical world. (Though I can’t be sure I haven’t missed anything.)
I mean, I agree it’d be evidence that alignment is hard in general, but “impossible” is just… a really high bar? The space of possible minds is very large, and it seems unlikely that the quality “not satisfactorily close to being aligned with humans” is something that describes every superintelligence.
It’s not that the two problems are fundamentally different it’s just that… I don’t see any particularly compelling reason to believe that superintelligent humans are the most aligned possible superintelligences?
I’m not sure either way on giving actual human beings superintelligence somehow, but I don’t think that not working would imply there aren’t other possible-but-hard approaches.
Until I actually see any sort of plausible impossibility argument most of my probability mass is going to be on “very hard” over “literally impossible.”
I mean, I guess there’s a trivial sense in which alignment is impossible because humans as a whole do not have one singular utility function, but that’s splitting hairs and isn’t a proof that a paperclip maximizer is the best we can do or anything like that.
I believe it is not literally impossible because… my priors say it is the kind of thing that is not literally impossible? There is no theorem or law of physics which would be violated, as far as I know.
Do I think AI Alignment is easy enough that we’ll actually manage to do it? Well… I really hope it is, but I’m not very certain.
I think we should not have very high confidence either way on most particular physical limits.
Aside from the fact that I just find this idea extremely hilarious, it seems like a very good idea to me to try to convince people who might be able to make progress on the problem to try. Whether literally sending Terry Tao 10 million dollars is the best way to go about that seems dubious, but the general strategy seems important.
I’d argue the sequences / HPMOR / whatever were versions of that strategy to some extent and seem to have had notable impact.
I mean I thought the entire point was to say it out loud, but if you want me to write it:
I no longer consent to being in a simulation.
i said the phrase and nothing happened
As I understand it, the word “qualia” usually refers to the experience associated with a particular sensation.
I don’t know how you can deny that people have “qualia” when, as far as I can tell, it was a word coined to describe a particular thing that humans experience?
While I think lesswrong provides notable value to me, I don’t think you can just divide the amount of value it’s given me by the number of days since I started reading content on it and say that each individual day is worth that much. If lesswrong were down once a month that wouldn’t remove 1⁄30 of the value of the website; I’d just read those articles on a different day.
While there’s definitely a cost to the front page being down for a day, the fact that I could just wait a day with next-to-no consequence makes it feel not very substantial. Maybe it’s different for other people?
If an aligned superintelligence can access most of the information I’ve uploaded to the internet, then that should be more than enough data to create a human brain that acts more or less indistinguishably from me—it wouldn’t be exact, but the losses would be in lost memories moreso than personality. Thus, I’m almost certainly going to be revived in any scenario where there is indefinite life extension.
This line of reasoning relies on a notion of identity that is at least somewhat debateable and I’m skeptical of it because it feels like rationalizing so I don’t need to change my behavior. But nonetheless it feels plausible enough to be worth considering.
The motivation to me seems exactly the same as with fiction: we’re talking about things other than physical objects or whatever.
To provide a more concrete example, I would say that the claim “There are at least three Jedi older than Anakin Skywalker” is, in the interpretation of the sentence almost anyone who understands it would use, a true statement, even though Jedi certainly do not exist in the same sense that New York City exists. I would not say that the fact that Jedi and New York exist in very different ways really violates Semantic Uniformity in any substantial way.
I definitely share your concern that evidence which isn’t “scientific” matters, but I still think whether or not there is scientific evidence isn’t entirely irrelevant to decision-making when we care about creating organizations that consistently make good decisions.
Currently, we definitely care far too much about scientific evidence, but I disagree that the concept is entirely bullshit.
Is the explanation in the linked sequence post insufficient?
It’s important to distinguish between different kinds of evidence. There’s never no rational evidence, but there might be no scientific evidence. The issue comes in when people conclude that there’s no rational evidence because there’s no scientific evidence.
I’d say lots of other things he’s said support that update. Stuff about how your model of the world will be accurate if and only if you somehow approximate Bayes’ law, for example.
The dath ilan based fiction definitely helped me internalize the idea better though.