@IL: Would I modify my own source code if I were able to? In this particular case, no, I wouldn’t take the pill.
Vladimir_Golovin
@IL: Of course, “I just feel that hurting living things is bad” sums the inner perspective quite well, but this isn’t really an answer to the question why exactly hurting living things feels bad for me, and why I wouldn’t take the pill that shuts down my mirror neurons.
By taking the pill, I create a people-hurter, a thing-that-hurts-people, which is undoubtedly a bad thing to do judging from the before-the-pill POV. It’s not that different from pressing a button that says “pressing this button will result in a random person being hurt or killed every day for 40 years since this moment”.
fugues, rainbows, stars, sunsets, and iridescent beetles
Yes, these are hard. Here’s my speculation:
Rainbows ⇒ sun and water ⇒ plants to eat ⇒ prey animals. Stars and sunsets ⇒ good weather tomorrow ⇒ easier survival (and reproduction). No idea about the beetles yet.
difficult to catch gazelles, the ugliness of easy to catch pigs, and the ugliness of tasty and nutritious bottom-dwelling fish.
These are easy.
All these things possess attributes desirable for a human. “I wish I could run as fast as a gazelle, or be as dangerous to my enemies as a tiger, or safely explore the land like an eagle”.
Pigs are prey, and this is not a desirable condition for a human. Plus, pigs are fat and smell bad. One can think, would I want to be fat, stinky and easy to catch?
Dogs are subordinate to us, wolves are not, therefore wolves possess more desirable attributes (they are not somebody’s slaves or sidekicks), which is why we see much more wolves than dogs on t-shirts and car stickers.
Wolves are pack predators, hyenas are too, but the latter eat mostly sick and dead animals—no human would want such a food. Wolves again are ‘more beautiful’ to us.
As for the bottom-dwelling fish, I personally find them strangely attractive and fascinating. Their ugliness might have to do with the fact that their ‘facial features’ can be easily projected to humans: huge mouth, widely spaced eyes, bald skin—not very attractive by human standards.
All this probably related to mirror neurons. We think pigs are ugly, but we don’t usually think that rocks are ugly.
As for the fugues, the reaction of the human brain to music is most puzzling and fascinating to me. It feels as if ‘some shadowy entity’ forgot to remove direct API for accessing our own emotional machinery.
Now it’s getting interesting. I finally understand what you were trying to say by your morality posts, which, I admit, I was unable to digest (I prefer to know where I’m going when I cross inferential distances). Please be sure you do a good post or two on your “Bayesian enlightenment”. I still vividly remember how profound was the impact of my own “Evolutionary enlightenment” on my earlier self.
I find this new style of writing intriguing.
Me too. I find it much more readable and enjoyable than the majority of previous posts.
Excellent post, agree with every single line of it. It’s not depressing for me—I went through that depression earlier, after finally understanding evolution.
One nitpick—I find the question at the end of the text redundant.
We already know that all this world around us is just an enormous pattern arising out of physically determined interactions between particles, with no ‘essence of goodness’ or other fundamental forces of this kind.
So the answer to your question seems obvious to me—if we don’t like patterns we see around us (including us ourselves, no exceptions), all we need to do is to use physics to arrange particles into a certain pattern (for example superintelligence) that, in future, produces patterns desirable for us. That’s all.
Eliezer, what’s your answer to the question?
I’m not an AGI researcher or developer (yet), but I think that the notion of a process steering the future into a constrained region is brilliant. It immediately feels much closer to implementation than any other definitions I’ve read before. Please continue posting on this topic. What I’m especially looking forward is anything on compression / abstraction over the search space.
I like the suggested hybrid idea (blog with fewer posts + community). Also, I definitely won’t mind the reduced post frequency—I’ll keep OB on my RSS list even if it goes down to one post per month.
Computers are quite good at painting, e.g. see the game Crysis.
They do that using dedicated hardware. Try to paint Crysis in realtime ‘per pixel’, using a vanilla CPU.
A question about Andrew Ng, who was mentined in this thread. Is that his real name?
Here’s his homepage: http://robotics.stanford.edu/~ang/
Looks like he’s one of the people who worked on the LittleDog robot: http://cs.stanford.edu/groups/littledog/
AGI Researcher: ”… I do agree that FAI is critically important.” ”… EY isn’t being very useful ATM.”
Isn’t this a contradiction, given that EY is one of the few people who publicly promote the idea of unfriendly AIs being fatal?
AGI Researcher: “There has been no /visible/ progress since [2004].”
What would you consider /visible/ progress? Running code?
Also, how about this: “Overcoming Bias presently gets over a quarter-million monthly pageviews”?
Eliezer, how about turning the original post into a survey? It’s already structured, so all that you (or someone with an hour of free time) have to do is:
1) Find a decent survey-creating site.
2) Enter all paragraphs the original post (maybe except #9) as questions.
3) Allow the results to be viewed publicly, without any registration.The answer to each question would be a list of radio-buttons like this:
( ) Strongly agree
(·) Agree
( ) Don’t know
( ) Disagree
( ) Strongly disagreeDoes anybody know a survey site that allows all of the above?
EY: you’ll find that it contains some tasks that are—there’s no kind word for this—effortful. (I would even say “difficult”, with the understanding that we’re talking about something that takes 10 minutes, not 10 years.)
Some tasks in WoW can take months to complete, and it’s clearly intended by WoW developers. Many tasks require ‘raiding’, which is an organized, coordinated activity involving up to 40 players, strategy, advance preparations, purchases, crafting etc. -- I have a friend who keeps a calendar of his evening raids and plans his real-world time in advance. When I played WoW, I didn’t raid at all because it placed too much constraints on my real-world schedule.
EY: So in the future, we’ll have programs that help you play the game
To a certain extent, we already do. Speaking of WoW again, we have Thottbot and Ludwig that help you instantly look up any item or spell (this function isn’t build into WoW), talent/glyph calculators, forums where people calculate all these probabilities of critical strikes, and a huge number of addons—for example Auctioneer, which lets people trade at the Auction House far more effectively (you see mean/median/average prices for a certain commodity, standard deviation, confidence ratings etc.), or Recount, which keeps stats about your fighting performance and displays them as graphs or pie charts.
Some of these ‘helper’ programs, however, are explicitly prohibited by the game vendors. Examples include WoW Glider, a bot that basically does the grindwork for you (it just repeatedly slays monsters to get you experience). Another example is from competitive FPS games—there is a kind of helper programs, called aimbots, that take the task of weapon aiming off the player, he just does the running. FPS vendors ban aimbots, and players absolutely hate anyone who uses them.
EY: So this is the ultimate end of the prophecy of technological progress—just staring at a screen that says “YOU WIN”, forever.
That definitely didn’t happen in modern games—the games that explored this area probably just didn’t sell, otherwise we’d see a lot more games of this kind. On the other hand, we have wireheads, such as alcoholics or drug addicts, who (seem to) do precisely that. The addiction issue is complicated by the physiological component of addiction. Speaking cynically, it would be interesting to see how many addicts abandoned the habit if their drug of choice was not physiologically addicting.
D. Alex:this is a bit out of left field, but: Is a setting like World of Warcraft a good medium for development of AI? Clear goals, clear measures of progress, sufficient complexity to provide an indication of when important insights are achieved, and a safe environment (in the sense that the path to paperclip AI seems unlikely)...?
Ben Goertzel proposes a exactly that (but for different reasons): http://www.kurzweilai.net/meme/frame.html?main=/articles/art0710.html
EY: So this is the ultimate end of the prophecy of technological progress—just staring at a screen that says “YOU WIN”, forever.
On second thought, playing a modern game IS staring at the “YOU WIN” screen.
Say, you just started playing a game. You did nothing at all, but you’re already immortal, you look badass, you have fists the size of a boulder, and you can engage some mean-looking bad guys and win!
So, the actual product of the game industry is 4D “YOU WIN” screens.
EY: The desire not to be optimized too hard by an outside agent is one of the structurally nontrivial aspects of human morality.
The vast majority of optimization-capable agents encountered by humans during their evolutionary history were selfish entities, squeezing their futures into their preferred regions. Given enough evolutionary time, any mutant humans who didn’t resist outside manipulation would end up ‘optimized’ to serve as slave labor in favor of the ‘optimizers’.
EY: would be the gift of a world that works on improved rules
Yes, just plug the most important holes (accidental death, unwanted suffering, illness, justice, asteroids, etc.), and leave people have fun.
Doug, I and my wife have always wondered about that. Some parts of the Russian folktales do sound pretty horrible today, for example, the widely-known Morozko begins with a woman marrying a widower with a daughter and forcing him to get rid of her (he agreed, took the daughter to the forest and left her there to freeze).
However, I’m not sure that the particular tale you posted is representative. Maybe it’s just one of the older stories that don’t make it into reprinting anymore. According to Vladimir Propp, the majority of Russian folktales have many structural elements similar to monomyth where the hero tends to cooperate with the helpers, and personally, I find many of these stories acceptable to read to my own daughter.
Personally, I don’t know what morality is, or what’s the ‘inherently the right thing to do’. For me, the situation is simple.
If I hurt someone, my mirror neurons will hurt me. If I hurt someone’s baby, I’ll experience the pain I inflicted upon the baby, plus the pain of the parents, plus the pain of everyone who heard about this story and felt the pain thanks, in turn, to their mirror neurons.
And I’ll re-experience all this pain in the future, every time I remember the episode—unless I invent some way to desensitize myself to this memory.
I’m a meat machine built by evolution. One of my many indicators of ‘inclusive genetic fitness’ is a little green light titled “I currently feel no pain”. If I hurt someone, this indicator will go red, which means that I will tend to avoid such behavior.
So, the short answer: I won’t be inclined to kill anyone even after some ‘authority’ tells me that killing is now ‘moral’ and ‘right’.
(A personal inside perspective: if I ever murder someone, I hope I’ll have enough guts to remove myself from the gene and meme pools—you can have my brain for cryo-slicing and my meat to feed some stray dogs).