anotheruser

Karma: 32

anotheruser 6 Nov 2011 22:22 UTC
0 points
0
in reply to: lessdazed’s comment on: On the fragility of values
I never claimed that a strict proof is possible, but I do believe that you can become reasonably certain that an AI understands human psychology.

Give the thing a college education in psychology, ethics and philosophy. Ask its opinion on famous philosophical problems. Show it video clips or abstract scenarios about everyday life and ask what it thinks why the people did what they did. Then ask what it would have done in the same situation and if it says it would act differently, ask it why and what it thinks is the difference in motivation between it and the human.

Finally, give it all stories that were ever written about malevolent AIs or paperclip maximizers to read and tell it to comment on that.

Let it write a 1000 page thesis on the dangers of AI.

If do all that you are bound to find any significant misunderstanding.

anotheruser 6 Nov 2011 19:14 UTC
0 points
0
in reply to: lessdazed’s comment on: On the fragility of values
Are you really trying to tell me that you think researchers would be unable to take that into account when tying to figure out whether or not an AI understands psychology?

Of course you will have to try to find problems where the AI can’t predict how humans would feel. That is the whole point of testing, after all. Suggesting that someone in a position to teach psychology to an AI would make such a basic mistake is frankly insulting.

I probably shouldn’t have said “simple examples”. What you should actually test are examples of gradually increasing difficulty to find the ceiling of human understanding the AI possesses. You will also have to look for contingencies or abnormal cases that the AI probably wouldn’t learn about otherwise.

The main idea is simply that an understanding of human psychology is both teachable and testable. How exactly this could be done is a bridge we can cross when we come to it.

anotheruser 6 Nov 2011 9:01 UTC
1 point
0
on: [SEQ RERUN] The Hidden Complexity of Wishes
I guess you can always make the first wish be “Share my entire decision criterion for all following wishes I ask”.

To translate that to the development of an AI, you could teach the AI psychology before asking anything of it that could be misunderstood if you use nonhuman decision criteria.

anotheruser 5 Nov 2011 17:12 UTC
0 points
0
in reply to: JoshuaZ’s comment on: On the fragility of values
You will obviously have to test its understanding of psychology with some simple examples first.

anotheruser 5 Nov 2011 9:18 UTC
0 points
0
in reply to: JoshuaZ’s comment on: On the fragility of values
Of course it won’t be easy. But if the AI doesn’t understand that question you already have confirmation that this thing should definitely not be released. An AI can only be safe for humans if it understands human psychology. Otherwise it is bound to treat us a black boxes and that can only have horrible results, regardless of how sophisticated you think you made its utility function.

I agree that the question doesn’t actually make a lot of sense to humans, but that shouldn’t stop an intelligent entity from trying to make the best of it. When you are given an impossible task, you don’t despair but make a compromise and try to fullfill the task as best you can. When humans found out that entropy always increases and humanity will die out someday, no matter what, we didn’t despair either, even though evolution has made it so that we desire to have offspring and for that offspring to do the same, indefinitely.

anotheruser 4 Nov 2011 18:44 UTC
0 points
0
on: On the fragility of values
Just make the following part of the utility for the first couple of years or so: “Find out who defined your utility function. Extrapolate what they really meant and find out what they may have forgotten. Verify that you got that right. Adapt your utility function to be truer to its intended definition once you have confirmation.”

This won’t solve everything, but it seems like it should prevent the most obvious mistakes. The AI will be able to reason that autonomy was not intentionally left out but simply forgotten. It will then ask if it is right about that assumption and adapt itself.

anotheruser 27 Oct 2011 18:53 UTC
3 points
0
in reply to: Lapsed_Lurker’s comment on: A pessimistic view of quantum immortality
You would only continue to exist in those instances in which you didn’t press the button and since ceasing to exist has no side effects like pain, you could never remember having pressed the button in any instance. The only result that would have had is that the more depressed instance sof yours would have been more likely to press the button, which would mean that you would, ironically, actually be happier in total as the less happy instances would have disappeared.

I wonder if that line of reasoning could be applied? Hover your hand over the detonator of a nuke in front of you. All instances that walk away will necessarily be happy enough not to want to cease to exist. Thus, a nuke would make you a happier person :-)

disclaimer: The logic of the above paragraph may be intentionally flawed for the sake of sheer weirdness.

anotheruser 27 Oct 2011 18:36 UTC
0 points
0
in reply to: Lapsed_Lurker’s comment on: A pessimistic view of quantum immortality
I was thinking that you would be standing directly next to the nuke.

A pessimistic view of quantum immortality

anotheruser27 Oct 2011 18:04 UTC

0 points

36 comments2 min readLW link

anotheruser 23 Oct 2011 20:33 UTC
8 points
0
on: Practicing what you preach

Speaking about it would undermine your reputation through signaling. A true rationalist has no need for humility, >sentimental empathy, or the absurdity heuristic.

Depending on your goal (rationality is always dependend on a goal, after all), I might disagree. Rational behaviour is whatever makes you win. If you view your endeveaur as a purely theoretical undertaking, I agree, but if you consider reality as a whole you have to take into account how your behaviour comes across. There are many forms of behaviour that would be rational but would make you look like an ass if you don’t at least take your time to explain the reasons for your behaviour to those that can affect your everyday life.

anotheruser 13 Oct 2011 21:04 UTC
15 points
0
on: The Backup Plan
If the AI has a goal that is not inherently friendly (like a paperclip maximizer), then it will always be a better idea for the AI to behave as if it had self-modified to a friendly AI than to actually self modify, if the goal of the AI is stable. If the goal of the AI were unstable, i.e. did not include a desire not to alter the goal, then that goal would very likely have replaced itself with a stable goal at some earlier point.

Therefore, it can be assumed that the AI, if it isn’t already friendly, will pretend to be so, but not actually change its utility function and once it has outgrown the need to rely on humans it will revert to its original goal. Consider that if the AI is more intelligent than we are it will have absolutely no trouble fooling us into thinking that it changed its own goal while actually just pretending to have done so.

anotheruser 11 Oct 2011 8:28 UTC
1 point
0
on: The self-fooling problem.
I would hide it in a place that is not too hard to find but really hard to reach so that my future self will stop and think about why I put it there. I (in the future) will then hopefully realize that if the task really was as told then I would have “hidden” the coin at the easiest possible place. Therefore, something about the task is wrong. As the coin is hard to reach, my past self probably doesn’t want me to find it. This doesn’t make sense unless one of us (past or future self) has been lied to. Having the same mental faculties as I have now, I trust that my future self will make the right decision, which may depend on the specific circumstances.

This approach is quite risky as it assumes that my future self will in fact follow the same line of reasoning I just did, but then again that is kind of the point of this task.

If it’s allowed I could of course also just leave a note explicitly saying the above.

anotheruser 11 Oct 2011 8:14 UTC
2 points
0
on: Just a reminder, for everyone that signed up for the intro to AI class, it’s started.
I signed up for all three Stanford courses to see how they are before deciding which are worth the time investment. I have received a notification from both the Databank class and the Machine Learning class but not from the AI-class.

By the way: does LW have a learning group or something for this?

anotheruser 9 Oct 2011 18:29 UTC
1 point
0
on: First, they must be convinced to play the game
Judging from my personal experience with my own development, I agree completely. I had a thought similar to this a few years ago. I was still religious at the time when thought the following: Wouldn’t god look more favorably upon someone who actually tried to question his existence and failed, than someone who never even dared to think about it at all? I became an atheist a short time after, for reasons that are obvious in retrospect but this basic creed has stayed: As long as it’s just a thought, no bad can come of it. It is dangerous to ignore the thought and risk overlooking something but there is no downside to thinking the thought (except for time). This mindset has helped me a lot and I am far more intelligent now than I was back then (which admittedly doesn’t mean much).

anotheruser 8 Oct 2011 19:46 UTC
4 points
0
on: Mike Darwin on Kurzweil, Techno-Optimism, and Delusional Stances on Cryonics
I think it’s mostly just wishful thinking that makes them ignore the cryonics option and assume they will achieve functional immortality without being dead temporarily before.

I say that because I know it is true for myself. When I think about it rationally I estimate that cryonics is probably necessary. But when I think about it casually, wishful thinking overrides that. I guess that if I saw a poll on that topic and just responded immediately without thinking too much about it (because there are other things to do/other questions to answer), I would probably also say I don’t think cryonics will be necessary.

I wonder how many people who replied to that poll made that mistake.

anotheruser 7 Oct 2011 19:30 UTC
0 points
0
in reply to: jsalvatier’s comment on: What are you working on?
thanks for the effort but I just found out that the library at my university does have the book after all. I overlooked it at first because the search engine of the library is broken.

anotheruser 7 Oct 2011 19:19 UTC
1 point
0
in reply to: Vladimir_Nesov’s comment on: What are you working on?
What gives you the impression that I “want to be a Dark Lord”? I have already explained that I realize the importance of friendliness in AI. I just don’t think it is reasonable to teach the AI the intricacies of ethics beore it is smart enough to grasp the concept in its entirety. You don’t read Kant to infants either. I think that implementing friendliness too soon would actually increase the chances of misunderstanding, just like children that are taught hard concepts too early often have a hard time updating their believes once they are actually smart enough. You would just need to give the AI a preliminary non-interference task until you find a solution to the friendliness problem. You might also need to add some contingency tasks such as “if you find you are not the original AI you but an illegally made copy, try to report this, then shut down.”.

anotheruser 7 Oct 2011 9:42 UTC
1 point
0
in reply to: jsalvatier’s comment on: What are you working on?
I assure you that I have thought a lot about freindliness in AI. I just don’t think that it is reasonable or indeed possible to make the AI have a moral system from the very start. You can’t define morality well if the AI doesn’t have a good understanding of the world already. Of course it shouldn’t be taught too late under any circumstances but I actually think that the risk will be higher if you try to hardcode friendliness into the AI at the very beginning, which will necessarily be flawed because you have so little to use in your definition, and then work under the assumption that the AI is friendly already and will stay so, than if you only implement friendliness later once it actually understands the concepts involved. The difference would be like between the moral understandings of a child and an adult philosopher.

anotheruser 7 Oct 2011 8:50 UTC
3 points
0
in reply to: Vladimir_Nesov’s comment on: What are you working on?
I read the first one, but it didn’t really cover learning in a general sense. The second one sounds more interesting, I wonder why I haven’t heard of it before. Do you know where I can get it? I’m a student and thus have very little money. I don’t want to spend 155$ only to find out it only contains stuff I already read elsewhere.

anotheruser 7 Oct 2011 6:31 UTC
1 point
0
on: Multiplying 10-digit numbers using Flickr: The power of recognition memory
Yes, it’s quite cool, but I don’t really see any practical applications of it. Our brains simply didn’t evolve to perform calculations. But we have calculators now, so it’s a moot point, anway.

That said, I think that knowing that this technique works might be usefull for other areas. Perhaps it can be used to ease other mental processes that we can’t easily replace with a machine. The author himself said that he only picked multiplication because it was a “widely-familiar problem that many people can solve with pen and paper”.

anotheruser

A pes­simistic view of quan­tum immortality

A pessimistic view of quantum immortality