I’ll replace it without the spacing so it’s more compact. Sorry about that, I’ll work on my comment etiquette.
Yorick_Newsome
Karma: 71
Maybe I’m wrong, but it seems most people here follow the decision theory discussions just for fun. Until introduced, we just didn’t know it was so interesting! That’s my take anyways.
Big Edit: Jack formulated my ideas better, so see his comment.
This was the original: The fact that the universe hasn’t been noticeably paperclipped has got to be evidence for a) the unlikelihood of superintelligences, b) quantum immortality, c) our universe being the result of a non-obvious paperclipping (the theists were right after all, and the fine-tuned universe argument is valid), d) the non-existence of intelligent aliens, or e) that superintelligences tend not to optimize things that are astronomically visible (related to c). Which of these scenarios is most likely? Related question: If we built a superintelligence without worrying about friendliness or morality at all, what kind of things would it optimize? Can we even make a guess? Would it be satisfied to be a dormant Laplace’s Demon?
I had a dream where some friends and I invaded the “Less Wrong Library”, and I agree it was most impressive. …in my dream.
^ Yossarian, a character in the novel Catch 22, by Joseph Heller.
I am probably in way over my head here, but…
The closest thing to teleportation I can imagine is uploading my mind and sending the information to my intended destination at lightspeed. I wouldn’t mind if once the information was copied the teleporter deleted the old copy. If instead of 1 copy, the teleporter made 50 redundant copies just in case, and destroyed 49 once it was confirmed the teleportation was successful, would that be like killing me 49 times? Are 50 copies of the same mind being tortured any different than 1 mind being tortured? I do not think so. It is just redundant information, there is no real difference in experience. Thus, in my mind, only 1 of the 50 minds matter (or the 50 minds are essentially 1 mind). The degree to which the other 49 matter is only equal to the difference in information they encode. (Of course, a superintelligence would see about as much relative difference in information between humans as we humans see in ants; but we must take an anthropocentric view of state complexity.)
The me in other quantum branches can be very, very similar to the me in this one. I don’t mind dying in one quantum branch all that much if the me not-dying in other quantum branches is very similar to the me that is dying. The reason I would like there to be more mes in more quantum branches is because other people care about the mes. That is why I wouldn’t play quantum immortality games (along with the standard argument that in the vast majority of worlds you would end up horribly maimed.)
If the additional identical copies count for something, despite my intuitions, at the very least I don’t think their value should aggregate linearly. I would hazard a guess that a utility function which does that has something wrong with it. If you had 9 identical copies of Bob and 1 copy of Alice, and you had to kill off 8 copies, there must be some terminal value for complexity that keeps you from randomly selecting 8, and instead automatically decides to kill off 8 Bobs (given that Alice isn’t a serial killer, utility of Alice and Bob being equal, yada yada yada.)
I think that maybe instead of minds it would be easier and less intuition-fooling to think about information. I also think that, like I said, I am probably missing the point of the post.
I’m slowly waking up to the fact that people at the Singularity Institute as well as Less Wrong are dealing with existential risk as a Real Problem, not just a theoretical idea to play with in an academic way. I’ve read many essays and watched many videos, but the seriousness just never really hit my brain. For some reason I had never realized that people were actually working on these problems.
I’m an 18 year old recent high school dropout, about to nab my GED. I could go to community college, or I could go along with my plan of leading a simple life working a simple job, which I would be content doing. I’m a sort of tabla rossa here: if I wanted to get into the position where I would be of use to the SIAI, what skills should I develop? Which of the ‘What we’re looking for’ traits would be most useful in a few years? (The only thing I’m good at right now is reading very quickly and retaining large amounts of information about various fields: but I rarely understand the math, which is currently very limiting.)
Point taken, I just think that it’s normally not good. I also think that maybe, for instance, libertarians and liberals have different conceptions of selfishness that lead the former to go ‘yay, selfishness!’ and the latter to go ‘boo, selfishness!’. Are they talking about the same thing? Are we talking about the same thing? In my personal experience, selfishness has always been demanding half of the pie when fairness is one-third, leading to conflict and bad experiences that could have been avoided. We might just have different conceptions of selfishness.
I liked this comment, but as anonym points out far below, the original blog post is really talking about “pre-scientific and scientific ways of investigating and understanding the world.”—anonym. So ‘just a few centuries ago’ might not be very accurate in the context of the post. The author’s fault, not yours; but just sayin’.
Oh jeeze, how did I miss that? Thanks for taking the time to enlighten me. About the ETA, I noticed that too, which may be relevant to another discussion I saw nested under the original quotation...
Did you read it in the context of the atheist blog post Eliezer linked to? I agree that the quote was possibly meant to be cautionary, but I think it was primarily meant to show that believing in things 200 years old is generally not a good idea. Maybe I misunderstood the point of the post, though; the cautionary value is a more useful interpretation for us aspiring rationalists, and ‘don’t put faith in ancient wisdom’ is rather simple advice by comparison. Because of that, context be damned (even if I did interpret it as was meant), I’m going to switch to your interpretation. :)
I think he was being sarcastic and trying to suggest that the original quote failed to take note that everyone thinks they are immune from those problems, including the person who decided the past was ‘wrong’ about them. I’m also pretty sure cousin_it is Russian, if that’s relevant. The USA thing was just a tasteful addition, the way I see it. I laughed. (His use of an exclamation point and a look at the top contributors list on the right also indicate sarcasm.)
Edit: I agree with Nick below. It was just a joke. Which I enjoyed.
“the quality of being selfish, the condition of habitually putting one’s own interests before those of others”—wiktionary
I can imagine a super giant mega list of situations where that would be bad, even if selfishness is often a good thing. There’s a reason ‘selfishness’ has negative connotations.
I’m not Eliezer, and perhaps not being an AGI researcher means that my answer is irrelevant, but I think that things can have a deep aesthetic value or meaning from which one could gain insights into things more important than AI or rationality. One of these things may be the ‘something to protect’ that Eliezer wrote about. Others may be intrinsic values to discover, to give your rationality purpose. If I could only keep one of a copy of the Gospels of Buddha or a copy of MITECS, I would keep the Gospels of Buddha, because it reminds me of the importance of terminal values like compassion. When I read GEB the ideas of interconnectedness, of patterns, and of meaning all left me with a clearer thought process than did reading Eliezer’s short paper on Coherent Extrapolated Volition, which was enjoyable but just didn’t seem to resonate in the same way. Calling these things ‘entertaining fluff’ may be losing sight of Eliezer’s 11th virtue: “The Art must have a purpose other than itself, or it collapses into infinite recursion.”
That is all, of course, my humble opinion. Maybe having everyone read about and understand the dangers of black swans and unfriendly AI would be more productive than having them read about and understand the values of compassion and altruism; for if people do not understand the former, there may be no world left for the latter.
Ah, thanks for the link. The only summit video I’ve seen before was Jurgen Schmidhuber’s, perhaps I should watch more of ’em.
I would repost this in the next open thread, it’s not like anyone would get annoyed at the double post (I think), and that site looks like it would interest a lot of people.
Perhaps there should be an ‘Open Thread’ link between ‘Top’ and ‘Comments’ above, so that people could get to it easily. If we’re going to have an open thread, we might as well make it accessible.
Anyways, I was looking around Amazon for a book on axiology, and I started to wonder: when it comes to fields that are advancing, but not at a ‘significant pace’, is it better to buy older books (as they’ve passed the test of time) or newer ones (as they may have improved on the older books and include new info)? My intuition tells me it’s better to buy newer books.
Well, if the people spitting out the clever non-sequiturs have charming British accents, then possibly. Otherwise, no… is Mensa about ‘debating’, normally? I always figured it’d be more of a casual social meet-up. But even then I suppose it could quickly dissolve into a mere signalling competition, or a ‘debate’.
That’s a good point. I’ve never actually interacted with someone in real life that even knew what philosophical zombies were, so my ‘intellectual’ conversations take place along the lines of ‘atheism versus theism’, sadly. Maybe there is some merit to joining Mensa after all?
I spent much of my childhood obsessing over symmetry. At one point I wanted to be a millionaire solely so I could buy a mansion, because I had never seen a symmetrical suburban house.