SIAI, Yudkowsky, Friendly AI, CEV, and Morality
This post entitled A Dangerous “Friend” Indeed (http://becominggaia.wordpress.com/2010/06/10/a-dangerous-friend-indeed/) has it all.
mwaser
Karma: −174
“Objective” good and bad require an answer to the question “good and bad for what?”—OR—“what is the objective of objective good and bad?”
My answer to that question is the same as Eli’s—goals or volition.
My argument is that since a) having goals and volition is good for survival; b) cooperating is good for goals and volition; and c) morality appears to be about promoting cooperation—that human morality is evolving down the attractor that is “objective” good and bad for cooperation which is part of the attractor for what is good for goals and volition.
The EXplicit moral basis that I am PROCLAIMING (not slipping through the back door) is that cooperation is GOOD for goals and volition (i.e. the morality of an action is determined by it’s effect upon cooperation).
PLEASE come back and comment on the blog. This comment is good enough that I will be copying it there as well (especially since my karma has been zeroed out here).
Yes. There are cases where occam’s razor results in a tie (or, at least, indistinguishably close).
And yet, the quantum mechanical world behaves exactly this way. Observations DO change exactly what happens. So, apparently at the quantum mechanical level, nature does have some way of knowing.
I’m not sure what effect that this has upon your argument, but it’s something that I think that you’re missing.
Do me a favor and check out my blog at http://becominggaia.wordpress.com. I’ve clearly annoyed someone (and it’s quite clear whom) enough that all my posts quickly pick up enough of a negative score to be below the threshold. It’s a very effective censoring mechanism and, at this point, I really don’t see any reason why I should ever attempt to post here again. Nice “community”.
Pushing a fat man onto the tracks to save several lives is generally considered to be immoral because you are USING a person to achieve some goal.
In your case, you are only using the trolley containing the man to stop the death of four people. You are NOT using the man because the trolley would work regardless of whether or not he is present. Thus, it is mere misfortune that he is present and killed—exactly as if he were on a siding where you diverted a train to save ten people.
I, too, really appreciated this post.
Unfortunately, though, I think that you missed one of the most important skills for safer reasoning—recognizing and acknowledging assumptions (and double-checking that they are still valid). Many of the most dangerous failures of reasoning occur when a normally safe assumption is carried over to conditions where it is incorrect. Diving three feet into water that is unobstructed and at least five feet deep won’t lead to a broken neck—unless the temperature is below zero centigrade.
Imagine that one day you come home to see your neighbors milling about your house and the Publisher’s Clearinghouse (PHC) van just pulling away. You know that PHC has been running a new schtick recently of selling $100 lottery tickets to win $10,000 instead of just giving money away. In fact, you’ve used that very contest as a teachable moment with your kids to explain how once the first ticket of the 100 printed was sold, scratched, and determined not to be the winner—that the average expected value of the remaining tickets was greater than their cost and they were therefore increasingly worth buying. Now, it’s weeks later, most of the tickets have been sold, scratched, and not winners and they came to your house. In fact, there were only two tickets remaining. And you weren’t home. Fortunately, your neighbor and best friend Bob asked if he could buy the ticket for you. Sensing a great human interest story (and lots of publicity), PHC said yes. Unfortunately, Bob picked the wrong ticket. After all your neighbors disperse and Bob and you are alone, Bob says that he’d really appreciate it if he could get his hundred dollars back. Is he mugging you? Or, do you give it to him?
Did you give the same answer to Omega? The cases are exactly analogous. (Or do you argue that they are not?)
A major upvote for this. The SIAI should create a sister organization to publicize the logical (and exceptionally) dangerous conclusion to the course that corporations are currently on. We have created powerful, superhuman entities with the sole top-level goal (required by LAW in for-profit corporations) of “Optimize money acquisition and retention”. My personal and professional opinion is that this is a far more immediate (and greater) risk than UnFriendly AI).
If Goertzel’s claim that “SIAI’s arguments are so unclear that he had to construct it himself” can’t be disproven by the simple expedient of posting a single link to an immediately available well-structured top-down argument then the SIAI should regard this as an obvious high-priority, high-value task. If it can be proven by such a link, then that link needs to be more highly advertised since it seems that none of us are aware of it.
Human evaluation of human values under specific instances is everything that Ben says it is (complex, nebulous, fuzzy, ever-shifting, and grokked by implicit rather than explicit knowledge).
On the other-hand, evaluation of a points in the Mandelbroit set by a deterministically moving entity that is susceptible to color-illusions is even more complex, nebulous, fuzzy, and ever-shifting to the extent that it probably can’t be grokked at all. Yet, it is generated from two very simple formulae (the second being the deterministic movement of the entity).
Eliezer has provided absolutely NO rational arguments (much less proof) that the core of Friendly is complex at all. Further, paying attention to the fact that ethical mandates within the obviously complex real world (particularly when viewed through the biased eyes and fallible beings) are comprehensible at all would seem an indication that maybe there are just a small number of simple laws underlying them (or maybe only one—see my comment on Ben’s post cross-posted at http://becominggaia.wordpress.com/2010/10/30/ben-goertzel-the-singularity-institutes-scary-idea/ for easy access).
Heh. I’ve read virtually all those links. I still have the three following problems.
Those links are about as internally self-consistent as the Bible.
There are some fundamentally incorrect assumptions that have become gospel.
Most people WON’T read all those links and will therefore be declared unfit to judge anything.
What I asked for was “an immediately available well-structured top-down argument”.
It would be particularly useful and effective if SIAI recruited someone with the opposite point of view to co-develop a counter-argument thread and let the two revolve around each other and solve some of these issues (or, at least, highlight the base important differences in opinion that prevent them from solution). I’m more than willing to spend a ridiculous amount of time on such a task and I’m sure that Ben would be more than willing to devote any time that he can tear away from his busy schedule.
Could I ask you to post the quotes as a separate post? They are priceless (and I’d love to be able to see what they applied to—so please include the references as well).
I don’t find that “truth” either obvious or true.
Would you say that “The obvious truth is that mind -design space contains every combination of intelligence and rationality”? How about “The obvious truth is that mind -design space contains every combination of intelligence and effectiveness”?
One of my fundamental contentions is that empathy is a requirement for intelligence beyond a certain point because the consequences of lacking it are too severe to overcome.
I clearly don’t understand karma
In the past, I have deleted negative scoring comments (long story but notice some of the odd scoring on some of my older posts). I have not deleted any at all recently.
It was my understanding, however, that karma never dropped below zero. Is what you’re saying that you build up a buffer where it will never go above zero either until all of your past negativity is paid off (which I would call going below zero but not being displayed as below zero—but that just semantics obviously)?
- Great!
So I should have taken the advice to come back as a new account after being gang-tackled (to use a polite term).
Kaj’s paper relies very heavily on Omohundro’s paper from AGI ’08. Check out the reply that I presented/published at BICA ’08 which (among other things) summarizes why the assumptions that Kaj relies upon are probably incorrect:
And I’d say . . . . “Sure! I recognize that I normally plan to finish 9 to 10 days early to ensure that I finish before the deadline and that I normally “fail” and only finish a day or two early (but still succeed at the real deadline) . . . . but now, you’ve changed the incentive structure (i.e. the entire problem) so I will now plan to finish 9 or 10 days before my new deadline (necessary to take your money) of 9 or 10 days before the real deadline. Are you sure that you really want to make that side bet?
I note also that “Would you care to make a side bet on that? is interesting as a potential conversation-filter but can also, unfortunately, act as a conversation-diverter.