Oh, I was unaware this was still an issue within this site. To LW the question of free will is already solved). I encourage you to look further into it.
However, I think our current issue can become a little more clear if we taboo “programming”.
What specific differences in functionality do you expect between “normal” AI and “powerful” AI?
are you asking what would I require to recognize some computing system as a “true AI”, or, basically, what is intelligence?
Somewhat. I think my question is better phrased as, “Why do you have a distinction between true intelligence and not true intelligence?”
My use of intelligence is defined (roughly) as cross domain optimization. A more intelligent agent is just better at doing lots of things it wants to do successfully, and conversely, something that’s better at doing a larger variety of tasks than a similarly motivated agent is considered more intelligent. It seems to me to be a (somewhat lumpy and modular) scale, ranging from a rock, up through natural selection, humans, and then a Superintelligent AI near the upper bound.
Hm. If this idea of intelligence seems valuable to you and worth pursuing, I absolutely implore that you wade through the reductionism sequence while or before you develop it more fully. I think it’d be an excellent resource for figuring out exactly what you mean to mean. (and the very similar Human’s guide to words)
Ah, that’s why I think reductionism would be very useful for you. Everything can be broken down and understood in such a way that nothing remains that doesn’t represent testable consequences. definitely read How an Algorithm Feels As the following quote represents what you may be thinking when you wonder if something is really intelligent.
Now suppose that you have an object that is blue and egg-shaped and contains palladium; and you have already observed that it is furred, flexible, opaque, and glows in the dark. [all the characteristics implied by the label “blegg”]
This answers every query, observes every observable introduced. There’s nothing left for a disguised query to stand for.
So why might someone feel an impulse to go on arguing whether the object is really a blegg [is truly intelligent]?
[brackets] are my additions.
The concept is sound, I think. Take an extreme example, such as the Gandhi’s pill thought experiment:
If you offered Gandhi a pill that made him want to kill people, he would refuse to take it, because he knows that then he would kill people, and the current Gandhi doesn’t want to kill people. This, roughly speaking, is an argument that minds sufficiently advanced to precisely modify and improve themselves, will tend to preserve the motivational framework they started in. (emphasis mine)
While it may be imperfect preservation while still stupid, or contain globular, fuzzy definitions of goals, an adequately powerful self improving AI should eventually reach a state of static, well defined goals permanently.
Yes, but the focus is on an already competent AI. It would never willingly or knowingly change its goals from its original ones, given that it improves itself smartly, and was initially programmed with (at least) that level of reflective smartness.
Goals are static. The AI may refine its goals given the appropriate information, if its goals are programmed in such a way to allow it, but it wont drastically alter them in any functional way.
An appropriate metaphor would be physics. The laws of physics are the same, and have been the same since the creation of the universe. Our information about what they are, however, hasn’t been. Isaac newton had a working model of physics, but it wasn’t perfect. It let us get the right answer (mostly), but then Einstein discovered Relativity. (The important thing to remember here is that physics itself did not change.) All the experiments used to support Newtonian physics got the same amount of support from Relativity. Relativity, however, got much more accurate answers for more extreme phenomena unexplained by Newton.
The AI can be programmed with Newton, and do good enough. However, given the explicit understanding of how we got to Newton in the first place (i.e. the scientific method), it can upgrade itself to Relativity when it realizes we were a bit off. That should be the extent to which an AI purposefully alters its goal.
For example, “We need to decide whether that AI is intelligent enough to just let it loose exploring this planet” implies a different definition of “intelligent” compared to, say, “We need to decide whether that AI is intelligent enough to be trusted with a laser cutter”.
Those sound more like safety concerns than inquiries involving intelligence. Being clever and able to get things done doesn’t automatically make something share enough of your values to be friendly and useful.
Better questions would be “We need to decide whether that AI is intelligent enough to effectively research and come to conclusions about the world if we let it explore without restrictions” or “We need to decide if the AI is intelligent enough to correctly use a laser cutter”.
Although, given large power (i.e. a laser cutter) and low intelligence, it might not achieve even its explicate goal correctly, and may accidentally do something bad. (i.e. laser cut a person)
one attribute of intelligence is the likelihood of said AI producing bad results non-purposefully. The more it does, the less intelligent it is.
Unless, of course, the machinery for caring is much simpler when it’s simply “care” vs “not care”. Pretending to care could be a much more complicated neurological adaptation that would be more wasteful than just implementing a nice “Sympathy” subsystem.
I mean, the way humans model each other’s behavior is by looking at our own self in other people’s scenarios, and then making minor adjustments for accuracy’s sake, since they think a little differently. I mean, why would you invent an entire subsystem just for understanding other people? That’s insane! YOU HAVE AN ENTIRE BRAIN ALREADY, AND EVERYBODY’S BRAIN IS REALLY DAMN SIMILAR, RIGHT?
Now, once you have this “self modeling and adjustment” system in place, actual caring makes a lot of sense. oh, pretending to care about your family and tribe is useful? here, i’ll just slap on an extra module here. we’ll call it Sympathy. It kinda works like this: you’v got that model of your brother you use to predict his behavior, right? It’s running on the same hardware YOUR mind is, and you care about YOUR mind, right? so we’ll just switch that “care” knob to the on position for your bro, alright? There. ha. now he’ll respond by viewing you as a valuable resource!
That “Really Intelligent Sociopath Who Accurately Predicts Every Scenario In Which Pretending To Care Is Useful And Rarely Makes Wasteful Mistakes Regarding That Decision” module seems a little bit costly to implement. Honestly, it’s just easier to actually freaking care about other people, albeit in a silly, ignorantly applied human way.
(I side with Harry on this one, in case you couldn’t tell.)
ok, since this is the stupid questions thread, how do you pronounce “knuth”? I really have no idea.
This may ease your anxiety regarding intelligence enhancement and enjoyment. perhaps not. I still recommend reading it.
The best solution to a problem is usually the easiest one.
-- GLaDOS from Portal 2
If best is defined as easiest, then the “usually” within the quote is entirely superfluous. “If” statements are logically exception-less, and the Law of Conserved Conversation (That i’ve just made up) means that “usually” implies exceptions. Otherwise it would be excluded from the quote. So I say, pedantically, “duh. but you’re missing the point a bit, aren’t you mate?”
I like to think of the principle as a kind of Occam’s for action. Don’t take elaborate actions to produce some solution that is otherwise trivially easy to produce.
Huh. The Maxim of Relation does sound very much like what I was trying to go for.
I’ve sent in mine, as I assume the position is still unfilled one week later. It’s by no means professional, but that’s correctable by mere equipment.
I wish everyone who’s submitted luck, and encourage any budding talents out there who remain on the fence to give it a try.
META COMMENT THREAD: KEEP ALL META DISCUSSION HERE
Anything you do or don’t like about this thread, post here. Feedback fosters improvement.
Original research? CONGRATU-FREAKIN-LATIONS!
Assuming, of course, that the whole things ends up being something cool.
+1 for doing a genuinely uncomfortable thing. Nice job.
Personal health habits, I like it.
I love my early morning lucid dreaming myself. I still need to work out the kinks in really controlling them, but even so, they’re pretty enjoyable.
As for charity, it should ABSOLUTELY be encouraged. It’s a less visceral awesome, of course, but I think we can all shut up and multiply if need be.
I think this is an example of reasoning analogous to philosophy’s “free will” debate. Human’s don’t have any more non-deterministic “free will” than a rock. The same is true of any AI, because an AI is just programming. It may be intelligent and sophisticated enough to appear different in a fundamental way, but it really isn’t.
It is posible for an optimizing process to make a mistake, and have an AI devolve into a different goal, which is what makes powerful AI look so scary and different. Example: Humans are more subject to each other’s whims than evolutionary pressures these days. Evolution has successfully created an intelligent process that doesn’t aim solely for genetic reproductive fitness. Oops, right?