Jonii

Karma: 513

Jonii Nov 10, 2010, 2:24 AM
0 points
in reply to: saturn’s comment on: Open Thread September, Part 3

Don’t forget that if it works, you probably get immortality too. If you were already immortal, would you be willing to become mortal for $500 000?

I’m not sure if this can be reversed just like that. If the immortality is possible at that world, you could just use part of that $500 000 to spring back to immortality, and I’m a bit unsure about how denying that affects our hypothetical situation and how it compares to the original dilemma.

But this would be quite close to the original for someone whose lifespan is almost over, so that money doesn’t have the time to change anything else for him. Still, one point that makes me wonder is that comparatively, we’d expect $500 000 be worth much less in the world where immortality is commonplace enough for you to have it, whereas now it’s really much. Do we assume that in this new world, $500 000 has same comparative edge as it would have in our world, or in other words, amount of money/people in the world remains the same?

Jonii Nov 9, 2010, 2:49 PM
1 point
in reply to: mwaser’s comment on: An Xtranormal Intelligence Explosion
My idea there was that if it’s not Friendly, then it’s not Friendly, ergo it is doing something that you would not want an AI to be doing(if you thought faster and knew more and all that). That’s the core of the quote you had there. Random intelligent agent would simply transform us into something of value, so we would most likely die very quickly. However, when you get closer to the Friendliness, Ai is no longer totally indifferent to us, but rather, is maximizing something that could involve living humans. Now, if you take an AI that wants there to be living humans around, but is not known for sure to be Friendly, what could go wrong? My answer, many things, as what humans prefer to be doing is rather complex set of stuff, and even quite little changes could make us really, really unsatisfied with the end result. At least, that’s the idea I’ve gotten from posts here like Value is Fragile.

When you ask if Friendliness is an attractor, do you mean to ask if intelligences near Friendly ones in the design spaces tend to transform into Friendly ones? This seems rather unlikely, as that sort of AI’s most likely are capable of preserving their utility function, and the direction of this transformation is not “natural”. For these reasons, arriving at the Friendliness is not easy, and thus, I’d say you gotta have some sort of a way to ascertain the Friendliness before you can trust it to be just that.

Jonii Nov 9, 2010, 1:08 AM
1 point
in reply to: JoshuaZ’s comment on: Open Thread September, Part 3
Sure, but I’ve gotten the impression that if someone mentions that they are not sure if the cryonics is worth the money, people come mentioning that “it’s actually only a year”, fallacy that I wanted to point out.

Jonii Nov 9, 2010, 12:14 AM
4 points
on: Open Thread September, Part 3
One thing people often seem to miss on LW, when discussing cryonics, is the cost of the operation. People seem to often operate under illusion that if the cost of process is, say, $50 000, you don’t need to worry about it that much since you can get a insurance, and thus pay only few hundred a year or so. This has made me wonder, since, insurance most likely makes it cost more, not less, and only works to offer protection from the case of you dying a lot earlier than insurance company would predict, which, you know, is unlikely.

This combined with the fact that even if you pay that cost, $50 000, you still are not guaranteed to be awakened in a better future. If the chance is ¹⁄₁₀, your expected utility is same as saving your life using surefire method for $500 000, and the chance is by most estimates lower than ¹⁄₁₀.

This is just to present my reasons for not getting cryonics deal even if I happened to live in a country where it would be realistically possible. Insurance doesn’t make your expected utility of signing up for cryonics any better, and even if you valued your life more than $50 000, it would be at least problematic to say if that was really the best you can do with all that money. Say you spent the same amount of a food that was a bit healthier than what you’re eating now, or paid someone to clean up your house and thus avoid stress regarding that and thus increasing your expected lifespan, it could easily be argued that you’re using that money better than our hypothetical cryonicists. And those are hardly the only and the best uses your $50 000 can have.

Of course, things get more difficult if you have only around 20 years or less to live. Still, I’m not sure if giving going through something that unlikely is going to help you and has huge costs is the best option.

Jonii Nov 8, 2010, 2:39 AM
11 points
in reply to: James_Miller’s comment on: An Xtranormal Intelligence Explosion

Not really. An AI that didn’t have a specific desire to be friendly to mankind would want to kill us to cut down on unnecessary entropy increases.

As you get closer to the mark, with AGI’s that have utility function that roughly resembles what we would want, but is still wrong, the end results are most likely worse than death. Especially since there should be much more near-misses than exact hits. Like, AGI that doesn’t want to let you die, regardless of what you go through, and little regard to your other sort of well-being, would be closer to the FAI than paperclip maximizer that would just plain kill you. As you get closer to the core of friendliness, you get all sorts of weird AGI’s that want to do something that twistedly resembles something good, but is somehow missing something or is somehow altered so that the end result is not at all what you wanted.

Jonii Nov 5, 2010, 9:18 PM
0 points
in reply to: timtyler’s comment on: The Curve of Capability
Dunno how to answer so I just stay quiet. I communicated badly there, and I don’t see fast way out of here.

Jonii Nov 5, 2010, 8:50 PM
9 points
on: The Curve of Capability
Even if AGI had only very small comparative advance(in skill and ability to recursively self-improve) over humans supported by then best available computer technology and thus their ability to self-improve, it would eventually, propably even then quite fast, overpower humans totally and utterly. And it seems fairly likely that eventually you could build fully artifical agent that was strictly superior(or could recursively self-update to be one) to humans. This intuition is fairly likely given that humans are not ultimately designed to be the singularity-survivors with best possible mindware to keep up with ever advancing technology, and re-engineering would most likely be more difficult than it was to build a new AI from scratch.

So in conclusion, nothing of a great importance, as far as I can tell, is changed. AGI that comes forward still has to be Friendly, or we’re doomed. And humans still are going to be totally overpowered by that machine intelligence in a relatively short time.

Jonii Nov 5, 2010, 8:24 PM
6 points
in reply to: timtyler’s comment on: The Curve of Capability
Your essay was the first thing that came to my mind after reading this post. I think the core argument is the same, and valid. It does seem to me that people vastly underestimate what computer-supported humans are capable of doing when AGI becomes more feasible, and thus overestimate how superior AGI actually would be, even after continously updating its own software and hardware.

Jonii Oct 25, 2010, 4:11 PM
1 point
on: Optimism versus cryonics
Nice post, though it avoided the reason why I don’t intend to get cryopreserved. That is, because it’s way too expensive.

Jonii Oct 25, 2010, 2:48 PM
2 points
on: The Problem With Trolley Problems
If you can come up with better hypotheticals, please do present one. The trolley-one is used so often most likely because there are no good alternatives.

Also, you seem to have missed the point of the trolley problem as it has been presented. The main point in thinking about it is that you notice that a), you have ethical intuitions, and b), they can change to polar opposite if the starting position is altered even a bit. That’s just healthy experimentation that’s supposed to provoke thoughts. I can’t see how changing this example would prevent people from justifying their thoughts in politics by misguided simplifications.

Jonii Sep 29, 2010, 7:32 AM
1 point
on: We have a new discussion area
lesswrong : discussion

a community forum devoted to refining the art of human rationality.

Jonii Sep 23, 2010, 6:07 AM
5 points
in reply to: byrnema’s comment on: Open Thread, September, 2010-- part 2
There is a proverb in go: “Lose your 100 first games as fast as possible”, mostly because there is little use in making plans before you have some sort of intuition of what’s going on on the board. So if you just play and lose without thinking too much about it, you get to the fun part faster.

Jonii Sep 22, 2010, 9:03 PM
1 point
in reply to: Morendil’s comment on: Open Thread, September, 2010-- part 2

The one with akrasia is, because I wouldn’t think of that as something you were learning.

I have dedicated much of my life for overcoming akrasia, especially after I found LW I’ve been working hard(well, thinking hard and planning hard mostly), does that count as learning?

Jonii Sep 22, 2010, 8:57 PM
0 points
in reply to: wnoise’s comment on: Open Thread, September, 2010-- part 2
Sure. I might comment here after I buy more of that.

Jonii Sep 22, 2010, 4:17 PM
0 points
in reply to: gwern’s comment on: Open Thread, September, 2010-- part 2
It would take longer, which is my main concern. Gotta see if this remains stabile for a reasonable amount of time until I can go testing around. For the last few weeks, it’s been far from stabile, I’ve remodeled my life just to be able to work more efficiently, and today is the first time I managed to actually work school-related stuff while being home, something I didn’t think was possible few months back(Before this, when I absolutely had to do something, I had to leave my apartment in order to do it).

Jonii Sep 22, 2010, 4:07 PM
0 points
in reply to: ata’s comment on: Open Thread, September, 2010-- part 2
Sorry, I don’t know. Omega 3, vitamins, stuff like that, but I threw the package away. It was some sort of combination box of all those supposedly important nutritients you need to take daily, with quite a lot of different stuff in it.

Jonii Sep 22, 2010, 4:03 PM
0 points
in reply to: gwern’s comment on: Open Thread, September, 2010-- part 2
I haven’t been taking those all that much lately, once every 4 days or so for the last two weeks.

Jonii Sep 22, 2010, 3:15 PM
2 points
in reply to: Jonii’s comment on: Open Thread, September, 2010-- part 2
Dunno if this is all that interesting story, but some people found it interesting, or at least inspiring, so I thought I should share. shrug

Jonii 22 Sep 2010 15:06 UTC
13 points
on: Open Thread, September, 2010-- part 2
So, hello. Has anyone here ever experienced spontaneous and sudden evaporation of akrasia alltogether? I’m asking mostly because this is exactly what happened to me, on 26th of August this year.

I didn’t do anything special. I had tried taking cold showers every now and then for a week earlier, and started taking some nutrient pills around that time, that’s pretty much it. Then, that morning, I suddenly started working on the projects I had planned and thought of.

That may not sound all that dramatic, but I haven’t introduced myself yet. I have been my whole life a rock-solid underachiever. After elementary school doing homework was not enforced, so I gradually stopped doing that. University doesn’t care if you participate in lectures, so I didn’t. All my academic effort happens roughly one day before any given exam. It’s not that I didn’t like the subject I study, or that I didn’t want to do it. I just couldn’t. There was a mental block that totally prevented me from using my free time for any of my projects, things I wanted to work on.

So that day, I spontaneously figured that I gotta study one thing in order to be prepared for the next academic year. So I did. I figured my room was suboptimal for studying, and I started cleaning it a bit. I figured I would achieve more elsewhere, so I went there and studied more. Then the night came, and I slept. Next day, this happened again. By now, I’ve completely reshaped my apartment so that I can work even at home, and I do work. I’m writing this after going to university, participating lectures like I should, and doing, at my free time, at my home, additional studying and work related to the most difficult subjects, in an organized order(Say, today it’s about SQL-stuff and Complex Analysis).

Basically, for my whole life, I have wished that I could just sit down and start working, for whatever reason, just because I felt like it was a good idea. Now I can, and it’s like a goddamn superpower.

But the scary and weird thing is, I have absolutely no idea how did this happen. I don’t feel that I have changed one bit, nor that I was doing things anyhow differently. If I traveled back to be me 2 months ago, I wouldn’t start working, all I could wish is that the miracle happened again in this new timeline also.

As a sidenote, this sort of weird leaps are not unheard of. I play go. Ratings in the game are based on winning percentage, so if you win against 4k 50% of time, you’re also 40, if there’s one standard deviation of difference, you’re either 3k or 5k, and so on. So basically, for 4k to win against 2k, there is about 2% chance(Is this right? I’m actually a bit unsure about exact numbers here). So, I started playing and after few months I encountered a block at 7k level. I kept on playing for a month or so, winning steadily about 50% of my games against 7k, and faring worse against stronger players. Then one morning, again, I started winning. It seemed I couldn’t lose against 7k players, nor 6k, or even 5k. Next day, I was playing at 3k level. And though I tried, I didn’t notice that anything had changed. It felt absurdly much that the rest of the world had just become weaker overnight. This was the most dramatic example, but even later on I have had similar leaps.

Jonii 18 Sep 2010 15:05 UTC
6 points
on: The conscious tape

Or the information on the tape, or—whatever it is that’s conscious, it is a static thing, not a living, dynamic thing. If computation is an output, process doesn’t matter. Time doesn’t enter into it.

Time is right where it should be in at: In the tape. We, the Matrix-Lords, are above the time of the tape, but it doesn’t mean the consciousness within the tape isn’t living and dynamic and all that.

The question presented is very intriguing, thank you for it.