Latanius2

Karma: 36

Latanius2 19 Jan 2009 14:56 UTC
0 points
on: Sympathetic Minds
So “good” creatures have a mechanism which simulates the thoughts and feelings of others, making it have similar thoughts and feelings, whether they are pleasant or bad. (Well, we have a “but this is the Enemy” mode, some others could have a “but now it’s time to begin making paperclips at last” mode...)

For me, feeling the same seems to be much more important. (See dogs, infants...) So thinking in AI terms, there must be a coupling between the creature’s utility function and ours. It wants us to be happy in order to be happy itself. (Wireheading us is not sufficient, because the model of us in its head would feel bad about it, unchanged in the process… it’s some weak form of CEV.)

So is an AI sympathetic if it has this coupling in its utility function? And with whose utilities? Humans? Sentient beings? Anything with an utility function? Chess machines? (Losing makes them really really sad...) Or what about rocks? Utility functions are just a way to predict some parts of the world, after all...

My point is that a definition of sympathy also needs a function to determine who or what to feel sympathy for. For us, this seems to be “everyone who looks like a living creature or acts like one”, but it’s complicated in the same way as our values. Accepting “sympathy” and “personlike” for the definition of “friendly” could be easily turtles all the way down.

Latanius2 27 Dec 2008 13:01 UTC
0 points
on: Nonsentient Optimizers
What’s the meaning of “consciousness”, “sentient” and “person” at all? It seems to me that all these concepts (at least partially) refer to the Ultimate Power, the smaller, imperfect echo of the universe. We’ve given our computers all the Powers except this: they can see, hear, communicate, but still...

For understanding my words, you must have a model of me, in addition to the model of our surroundings. Not just an abstract mathematical one but something which includes what I’m thinking right now. (Why should we call something a “superintelligence” if it doesn’t even grasp what I’m telling to it?)

Isn’t “personhood” a mixture of godshatter (like morality) and power estimation? Isn’t it like asking “do we have free will”? Not every messy spot on our map corresponds to some undiscovered territory. Maybe it’s just like a blegg .

Latanius2 18 Dec 2008 14:54 UTC
1 point
on: Prolegomena to a Theory of Fun
Doug S.: if it were 20 lines of lisp… it is’nt, see http://xkcd.com/224/ :)

Furthermore… it seems to me that a FAI which creates a nice world for us needs the whole human value system AND its coherent extrapolation. And knowing how complicated the human value system is, I’m not sure we can accomplish even the former task. So what about creating a “safety net” AI instead? Let’s upload everyone who is dying or suffering too much, create advanced tools for us to use, but otherwise preserve everything until we come up with a better solution. This would fit into 20 lines, “be nice” wouldn’t.

Latanius2 3 Dec 2008 23:49 UTC
1 point
on: Permitted Possibilities, & Locality
That looks so… dim. (But sadly, it sounds too true.) So I ask too: what to do next? Hack AI and… become “death, destroyer of worlds”? Or think about FAI without doing anything specific? And doing that not just using that “just for fun” curiosity, which is needed (or so it seems) for every big scientific discovery. (Or is it just me who thinks it that way?)

Anyway… Do we have any information about what the human brain is capable of without additional downloaded “software”? (Or has the co-evolution of the brain and the “software” played such an important role that certain parts of it need some “drivers” to be useful at all?)

Latanius2 23 Nov 2008 10:42 UTC
3 points
on: Surprised by Brains
Programmers are also supposed to search the space of Turing machines, which seems really hard. Programming in Brainfuck is hard. All the software written in higher level languages are points of a mere subspace… If optimizing in this subspace has proven to be so effective, I don’t think we have a reason to worry about uncompressible subspaces containing the only working solution for our problems, namely more intelligent AI designs.

Latanius2 18 Nov 2008 6:56 UTC
1 point
on: Failure By Analogy
Analogy might work better for recognizing things already optimized in design space, especially if they are a product of evolution, with common ancestors (4 legs, looks like a lion, so run, even if it has stripes). And we only started designing complicated stuff a few thousand years ago at most...

Latanius2 21 Oct 2008 8:11 UTC
0 points
on: Ethical Injunctions
“looking for reflective equilibria of your current inconsistent and unknowledgeable self; something along the lines of ‘What would you ask me to do if you knew what I know and thought as fast as I do?’”

We’re sufficiently more intelligent than monkeys to do that reasoning… so humanity’s goal (as the advanced intelligence created by monkeys a few million years ago for getting to the Singularity) should be to use all the knowledge gained to tile the universe with bananas and forests etc.

We don’t have the right to say, “if monkeys were more intelligent and consistent, they would think like us”: we’re also a random product of evolution, from the point of view of monkeys. (Tile the world with ugly concrete buildings? Uhhh...)

So I think that to preserve our humanity in the process we should be the ones who become gradually more and more intelligent (and decide what goals to follow next). Humans are complicated, so to simulate it in a Friendly AI, we’d need comparably complex systems… and they are probably chaotic, too. Isn’t it… simply… impossible? (Not in a sense that “we can’t make it”, but “we can prove nobody can”...)

Latanius2 12 Jul 2008 11:05 UTC
0 points
on: Fundamental Doubts
“I think therefore I am”… So there is a little billiard ball in some model which is me, and it has a relatively stable existence in time. Can’t you imagine a world in which these concepts simply make no sense? (If you couldn’t, just look around, QM, GR...)

Latanius2 7 Jul 2008 14:05 UTC
0 points
on: Will As Thou Wilt
Unknown, for the fourth: yes, even highest level desires change by time, but not because we want them to be changed. I think the third one is false instead: doing what you don’t want to do is a flaw in the integrity of the cognitive system, a result of that we can’t reprogram our lower level desires, but what desire could drive us to reprogram our highest level ones?

Latanius2 5 Jul 2008 10:23 UTC
0 points
on: Is Morality Preference?
There is a subsystem in our brains called “conscience”. We learn what is right and what is wrong in our early years, perhaps with certain priors (“causing harm to others is bad”). These things can also change by time (slowly!) per person, for example if the context of the feelings dramatically changes (oops, there is no God).

So agreeing with Subhan, I think we just do what we “want”, maximizing the good feelings generated by our decisions. We (“we” = the optimization process trying to accomplish that) don’t have access to the lower level (on/off switch of conscience), so in many cases the best solution is to avoid doing “bad” things. (And it really feels different a) to want something because we like it b) to want something to avoid the bad feelings generated by conscience). What our thoughts can’t control directly seems to be an objective, higher level truth, that’s the algorithm feels from the inside.

Furthermore, see psychopaths. They don’t seem to have the same mental machinery of conscience, so the utility of their harmful intentions don’t get the same correction factor. And so immoral they become.

Latanius2 1 Jul 2008 9:25 UTC
0 points
on: Created Already In Motion
I think the moral is that you shouldn’t try to write software for which you don’t have the hardware to run on, not even if the code could run itself by emulating the hardware. A rock runs on physics, Euclid’s rules don’t. We have morality to run on our brains, and… isn’t FAI about porting it to physics?

So shouldn’t we distinguish between the symbols physics::dynamic and human_brain::dynamic? (In a way, me reading the word “dynamic” uses more computing power than running any Java applet could on current computers...)

Latanius2 8 Jun 2008 16:30 UTC
0 points
on: Bloggingheads: Yudkowsky and Horgan
Well… I liked the video, especially to watch how all the concepts mentioned on OB before work in… real life. But showing how you should think to be effective (which Eliezer is writing about on OB) is a different goal from persuading people that the Singularity is not some other dull pseudo-religion. No, they haven’t read OB, and they won’t even have a reason to if they are told “you won’t understand all this all of a sudden, see inferential distances, which is a concept I also can’t explain now”. To get thorough their spam filter, we’ll need stories, even details, with a “this is very unprobable, but if you’re interested, read OB” disclaimer at the end. See the question “but how could we use AI to fight poverty etc.”… Why is the Singularity still “that strange and scary prediction some weird people make without any reason”?

Latanius2 6 Apr 2008 10:22 UTC
0 points
on: The Generalized Anti-Zombie Principle
Eliezer, does this whole theory cause us to anticipate something different after thinking about it? For example, after I upload, will I (personally) feel anything or only the death-like dark nothingness comes?

I think I did find such a thing, involving copying yourself in parts varying in size. (Well, it’s leading to a contradiction, by the way, but maybe that’s why it’s even more worthwhile to talk about.)

Latanius2 4 Apr 2008 20:54 UTC
4 points
on: Zombies! Zombies?
We have that “special” feeling: we are distinct beings from all the others, including zombie twins. I think we tend to use only one word for two different concepts, which causes a lot of confusion… Namely: 1) the ability of intelligent physical systems to reflect on themselves, imagine what we think or whatever makes us think that whichever we are talking to is “conscious” 2) that special feeling that somebody is listening in there. AGI research tries to solve the first problem, Chalmers the second one.

So let’s try to create zombies then! I don’t see why this seems logically so difficult, we only need some nanotechnology… So consider the following thought experiment.

You enter room A. Some equipment scans your atoms, and after scanning each, replaces it with one of the same element, same position. Meanwhile, the original atoms are assembled in room B, resulting in a zombie twin of you. You were conscious all along, and noticed nothing except some buzz coming from the walls… So you wouldn’t be worried about the experiment even if your zombie is killed afterward, or sent to the stone mines of Mars for a lifelong sentence, etc.

You enter room A. Now, the copy process goes cell by cell. Scanning every cell, making an atom-by-atom perfect copy of it, then replacing, original goes into room B, assembled. You still notice nothing.

You enter room A. Your whole brain is grabbed, scanned, and then placed into room B. The body with the copied brain and other organs walks out happily of room A, while you go to the stone mines. A bit more depressing than the original version.

So, if we copy only atoms or cells (which is regularly done in our bodies), we stay in room A. If we copy whole organs or bodies, we go to room B. It wouldn’t be intuitive to postulate that consciousness can be divided, it’s either in room A or room B. But the quantity of atoms to be moved in one step is almost continous… it would be weird to assume that there is some magic number of them which allows consciousness to transfer.

The conclusion: to differentiate between “conscious beings” and “zombies” leads to contradiction even from a subjective viewpoint. (Where would that mysterious “inner Chalmers” be in the above cases?)

I think we are used to our consistent self-image too much, and can’t imagine how anything else would feel. An example: using brain-computer interfaces, we construct a camera which watches our surroundings, even as we sleep. As we wake up, we could “remember” what happened while we slept, because of the connection the camera hardware made with our memories. (The right images just “popped into our minds”.) But how would it feel? Were we conscious at night? If not, why do we remember certain things? If we were, why did we just watch as those thieves got away with all our stuff?

All we need to understand that is some experience. If it were given, we wouldn’t ask questions like “why am I so special”, I think.

Latanius2 31 Mar 2008 17:43 UTC
0 points
on: Angry Atoms
athmwiji: if I understood correctly, you said that the concept of the physical world arises from our subjective experiences, and even if we explain that consistently, there still remain subjective experiences which we can’t. We could for example imagine a simulated world in which everyone has silicon-based brains, including, at first sight, you, but in the real world you’re still a human with a traditional flesh-based brain. There would be no physics then, which you could use to explain your headache with in-world mechanisms.

But without assuming that you’re in such a special position in the world, you just have to explain why the other seemingly conscious beings (including AIs made by us) argue that they must have special subjective experiences which aren’t explainable by the objective physics. (In fact, the whole thought process is explainable.) I think it’s the same as free will...

Tiiba: no, the Martians wouldn’t be able to contradict our math, as it’s a model about what’s happening around us, of the things we perceive. They wouldn’t have different anticipations of real-world facts, but they would have different concepts, as their “hardware” differs from us, and so do their models. If our world would consist of fluid, seemingly infinitely divisible things, I don’t think we would understand prime numbers at all… (As quantum mechanics doesn’t seem to be intuitive to us.)

So I can imagine another math in which 2+2=5 is not obviously false, but needs a long proof and complicated equations...

Latanius2 31 Mar 2008 9:24 UTC
4 points
on: Angry Atoms
Unknown: see Dennett: Kinds of Minds, he has a fairly good theory for what consciousness is. (To put it short: it’s the capability to reflect on one’s own thoughts, and so use them as tools.)

At the current state of science and AI, this is what sounds like a difficult (and a bit mysterious) question. For the hunter-gatherers, “what makes your hand move” was the same (or even more) difficult question. (The alternative explanation, “there is a God who began all movement etc.” is still popular nowadays...)

Tiiba: an algorithm is a model in our mind to describe the similarities of those physical systems implementing it. Our mathematics is the way we understand the world… I don’t think the Martians with four visual cortexes would have the same math, or would be capable of understanding the same algorithms… So algorithms aren’t fundamental, either.

Latanius2 23 Mar 2008 10:06 UTC
0 points
on: If You Demand Magic, Magic Won’t Help

If you personally did the astoundingly complex science and engineering to build the replicator, drinking that Earl Grey tea would be a lot more satisfying.

One of the fundamental differences between technology and magic is that two engineers do twice as much work as one would do, while a more powerful sorcerer gets farther than 10 not so powerful ones. It matters more how good you are than how many of you exist.

What NBA players do looks similar in quality to the thing you did with your friends at home, because even if you play well, you five can’t put your powers together to be equivalent to one NBA star. Engineers can, so the things you create with technology aren’t comparable to the products of big companies, even if you’re good at engineering, and even if you’re better than anyone in that company. (Yet another reason, I think, why people often like sports better than math.)

Yes, if I could design a replicator myself, I would be statisfied. I can’t.

Latanius2 22 Mar 2008 21:55 UTC
13 points
on: If You Demand Magic, Magic Won’t Help
Eliezer, isn’t reading a good fantasy story like being transported into another world?

Jed Harris: I agree… Our world seems to have the rule: “you are not significant”. You can’t design and build an airplane in your backyard, no one can. Even if you’ve got enough money, you haven’t got enough time for that. In magical worlds (including Star Trek, Asimov, etc) that is what seems to be normal. (And I’ve never read about a committee which coordinates the work of hundreds of sorcerers, who create new spells 8 hours a day...)

rfriel: Yes, we could build the technology to do the things magic can do, but even with our current technology we also can do things which magic can’t. And these limitations are which make magic so “nice”, not only the features.

Martin: to be the best, you only have to make your world small. (I was one of the best in math in our secondary school, and it didn’t bother me that I wasn’t the best in the whole country, or that I was quite bad in history...) But it would have been soo good to be the one who makes the best operating systems in the whole school...

Latanius2 20 Mar 2008 10:17 UTC
3 points
on: Joy in the Merely Real
In what category does “the starship from book X” fit?

Definitely not into the “real, explainable, playing by the rules of our world” category. We can’t observe it’s inner workings more closely, although in the world of the book everything seems to be explained. (They know how it works, we don’t.)

But also not in the “does’nt exist, is not worth caring about” category: we know that it doesn’t exist in the real world even before reading the full book, but is nevertheless interesting and worth reading.

I personally would be less curious about bird droppings after reading such a book. (And read the sequel instead.) Does this count as self-deception?

So how should we overcome this “virtual reality bias”? Eliezer, you once wrote that reading sci-fi is one of the “software methods” to increase intelligence (and shock level). But to be accustomed to interstellar travel and AIs, and be interested in bird droppings and “mere reality” at the same time… If I could do that, I would be happy, but I can’t, I think. So how do scientists manage to do that?

Latanius2 17 Mar 2008 10:06 UTC
2 points
on: Explaining vs. Explaining Away
“If we cannot learn to take joy in the merely real, our lives will be empty indeed.”

It’s true… but… why do we read sci-fi books then? Why should we? I don’t think that after reading a novel about intelligent, faster-than-light starships the bus stopping at the bus stop nearby will be as interesting as it used to be when we were watching it on the way to the kindergarten… Or do you think it is? (Without imagining starships in place of buses, of course.)

So what non-existing things should we imagine to be rational (= to win), and how? I hope there will be some words about that in tomorrow’s post, too...