adamzerner
Thank you for this. The idea of “if you die before the singularity but are signed up for cryonics you might be revived” didn’t really register with me until now. I feel silly. It’s a hugely important thing that I overlooked. It’s kinda shaken me. Currently, avoiding death is a pretty big thing for me, but given this, it may not be something worth prioritizing so much. Let me try to play with some numbers.
I suppose we’re just multiplying by the probability of immortality without resurrection. Eg. if I die right now, let’s ignore the ~50 years of pre-singularity life I lose and focus on me losing the 10% chance of living 100k post-singularity years. Or an expectation of 10k years. But I only lose those 10k years if it’s immortality without resurrection. So what is the probability of immortality without resurrection? Suppose it’s 30%. Then the expectation is 3k years instead of 10k.
Furthermore, if it’s immortality without resurrection, I think those life years are more likely to be unpleasant. I might not even want to be living those life years. Doesn’t immortality without resurrection indicate pretty strongly that the AI is unfriendly? In which case, it wouldn’t make sense to go to great lengths trying to avoid death, eg. by not riding in cars.
On the other hand, when people die in car accidents, it seems like the type of thing where your brain could be damaged enough such that you wouldn’t be able to be cryonically frozen. Hm, this feels pretty cruxy. There’s gotta be at least a 10% chance that the car accident that kills you would also prevent you from being cryonically frozen, right? If so, we’re only cutting things down by an order of magnitude. That seems like a lower bound. In realize, I’d think that it’s more like a 50% chance.
Good points, maybe you’re right.
Suppose we use the assumptions I made in my previous comment of death costing $1B and injury costing $5M. And suppose we ballpark it and say that injuries are 100x more likely than death. That makes it more like $1B to $500M. However, that is assuming all injuries cost $5M, and that seems high. It assumes that a) it is a permanent injury and b) it cuts the value of life fully in half. I’d ballpark it at more like maybe $1M on average. In which case it’s more like $1B to $100M.
But this is comparing injury costs to the $1B number for the value of life, which you could argue is high. If you disagree with the $1B number, then injuries seem like they might be important. For example, if we forget about the whole life extension thing and use the assumption of $10M for the value of life and $1M for the value of avoiding injury, with injury being 100x as likely as death, the injury component becomes the bigger factor.
But this extra injury stuff probably doesn’t change driving from “worth it” to “not worth it”. So the crux of the question still does seem to revolve around how much you value life.
Interesting, thanks! I think that for now I’m gonna stay in the US because of a few practical things: jobs, friends/family, and language. In the future I could see myself moving outside the US though.
ONE: I love how “should I learn to drive for this trip right here?” cascades into this vast set of questions about possible future history, and AGI, and so on <3
Yeah, it is interesting isn’t it. Personally I actually would prefer if it were a more mundane decision, like whether or not I want to deal with the traffic or something :)
Another great place for linking “right now practical” questions with “long term civilizational” questions is retirement. If you have no cached thoughts on retirement, you might profitably apply the same techniques used for car stuff to “being rich if or when the singularity happens” and see if either thought changes the other?
Wow, this is a very good point. Thank you! I thought about it briefly in the past, but it hadn’t hit me until now that similar logic of “small chance of moderate utility over an extremely long time horizon” might apply to things like “be rich when the singularity happens”. I always had imagined that singularity = amazingness forever for everyone, but I think my inner Professor Quirrell must have fell asleep. He’s awake now and is yelling at me.
Do you have any thoughts in particular on this? For me personally I plan on being rich anyway so that I can fund FAI research or something, but I still think it’s worth thinking about.
TWO: I used to think “I want to live this year”, “If I want to live in year Y then I will also want to live in year Y+1″. Then by induction: “I will want to live forever”.
My thought here is that the post-singularity future will be a black swan, and that this natural decay you’re describing wouldn’t apply. It’ll be an amazing place to live where people won’t want to die. Hopefully.
I know there might be normal things about human psychology that would make people perhaps get bored and stop enjoying life at eg. their 10,000th birthday, but in a post-singularity world, it seems quite likely that we’d be able to overcome this. I guess that gets a little wire heading-y, but it’s not excessive to my taste.
Maybe I’m not thinking about this properly, but it seems to me like more of a yes-or-no sort of thing than a progressive decay sort of thing. Eg. it doesn’t get more likely that people will want to die as time goes on in the post-singularity world. It’s just a question of whether the post-singularity world has solved that problem or not.
caused me to be interested in authentic durable happiness, in general, in humans, and also a subject I invented for myself that I call “gerontopsychology”
I have a weird feeling you’d be interested in Michael Plant of the EA community’s work on “ordinary human unhappiness”. It’s always something I’ve thought about too as a worthwhile idea to pursue.
THREE: I drive <3
Haha, to each their own :)
Yeah I agree. This is one of the places where I could see orders of magnitude differences that are strong enough to drive changes in the ultimate conclusion. Eg. from “it isn’t worth the risk” to “it is worth the risk”. I’ve only been able to do a very handwavvy estimate of you having 1⁄4 of the risk if you’re safe. Josh Jacobson’s analysis looks like it’s closer to 1⁄6. His factored in seatbelt wearing, but not speed limit, and speed limit does make a lot of sense.
It’d also be great if there was good data on more obscure things like wearing a helmet in a car, although iirc a helmet specifically might actually cause more harm due to the torque causing worse whiplash. Anyone know if that’s true?
Update: It looks like urban compared to rural only reduces fatality rate by about a factor of two. However, you probably have to drive more miles if you’re in a rural place, so maybe urban can get up to a 10x improvement.
Hm, the reason I went with cost per year is that it felt like it would be easier to think about that way. Eg. $1000/year vs $3.33/hr, but now that I’m thinking about it again that actually doesn’t seem to be the case. At all. Good point.
Thanks for the tip on Uber Black. Sorry to hear about the accidents.
The future is inherently uncertain, so if I were to take a decreased quality of life (no more driving) to better my chances of surviving to ASI, I had better have strong intuition that ASI would come.
Yeah, I definitely hear ya. I have these feelings too. But at the same time, I think it’s in violation of Shut Up and Multiply.
My main worry about decreasing my quality of life by not driving / decreasing risk is the alignment problem for AI. I can image a counterfactual world where AI is created but it is not aligned, where if I were to take heavy precautions I would suffer decreased quality of life many years just to die to a rouge agent.
I hear ya here too. It’s once of the main places that affects the conclusion, I think. My reasoning for expecting a post-singularity year to have positive utility is because it seems like a place where it’d make sense to adopt the opinion of the experts I quoted in the article. (And that it’s less depressing.)
I didn’t quite phrase it like this, but I’m thinking that ultimately the question is about whether driving is worth it per unit of time. So even if I drive less often, the question is still “Is it worth it for me to drive these 20 miles?”. Also, I meant for the question to be more general, not necessarily whether I personally should or should not drive.
Around half deaths from car accidents are pedestrians (may be less in US). By choosing not to drive, you increase the time of walking and your chances of being hit by other person’s car.
My impression is that if you don’t drive you wouldn’t be covering the same distances as you would in a car. Eg. if you walked 5 miles vs drove 5 miles, maybe the risk of being hit as a pedestrian starts to add up, but since in practice you’d only be walking a few blocks, the risk of death would be many times less. Similar point for biking.
Also, maybe this is overconfidence, but I feel like you have more control over your risk of death when walking or biking. Eg. you can be really careful about crossing the street, whereas on the road there’s only so much you can do. For example, if the crosswalk thing turns green but there is a car coming fast such that it’d have to stop in order to avoid hitting you as you cross the cross walk, I say you shouldn’t cross the crosswalk even though you’re “supposed” to. Maybe my memory is being overly dramatic, but I’m pretty sure I’d be dead right now if I didn’t adopt this policy.
Finally, some cars are like two-three orders safer than others, if we look at the number of reported deaths per billion km driving. I saw once that Toyota Prius had 1 death for 1 billion km, but Kia Rio was only 1 for 10 millions.
That example in particular doesn’t control for Prius drivers potentially having a more cautious personality or something than Rio drivers. However, it is still quite telling. If cars really do differ this much in safety then that is a very important point. Do you or anyone else have more info about this? I feel some skepticism about it, but am ready to update!
Waiting few years for self-driving Tesla Cybertrack may be an option.
In theory I think this is beyond the scope of the question, but in practice I think it’s a good point. It’s nice to know if driving might become something that is worth the risk in the future.
FWIW, I did have this question in mind when I asked Is driving worth the risk?. I chose not to ask this question because it is sort of a beast to answer, whereas the driving question gets at this question while keeping things ballpark-y, approachable, and relatable, which seems like it makes it a good first step.
I agree that pruning down the list of expected outcomes is a simplification. However, it seems like the badness is dominated by the outcome of dying enough where the other outcomes become negligible, and thus such a simplification is appropriate.
For example, suppose we use $100k as the value of a post-singularity year, 100k years as the expected post-singularity lifespan, and 10% as our chance of reaching the singularity. Those assumptions give us a value of life of $1B, which makes the death component of the EV calculation of driving -$25k. Now compare that to the injury component. For a lifelong injury, maybe it reduces the value of life from $200k to $100k, and you expect to live another 50 years. So the injury costs $5M versus death costing $1B. Getting permanently injured would have to be many, many times more likely than dying in a car accident for it to start becoming non-trivial next to the death component, with these assumptions.
Descriptively I know that hyperbolic discounting is a thing. But prescriptively, it’s not clear to me that it should be. Do you mind elaborating on why you think it should be?
Also, I think this only really starts to change the conclusions we draw when the discounting is significant, eg. a post-singularity year going from $100k to $10k rather than $100k to $90k. It sounds like you are saying it should be significant. Is that true? Why is that?
Using the $10 million value for your life
That makes enough sense, but the $10M valuation of life seems like it’s the cruxiest part of the question.
Given that amount, I should be comfortable driving (a maximum of) 264 miles each week, as they provide the same risk.
I’ve been thinking about this recently. I’ve been pretty risk averse when it comes to covid. There are probably things I do, like driving, that are riskier than the risks I avoid due to covid. You can say, “well you’re willing to eg. drive, so you should also be willing to take these covid associated risks”.
The issue with this logic is that revealed preferences aren’t real. Humans aren’t consistent and don’t always make good decisions. In your case, if the conclusion of this investigation into driving says “it’s a terrible idea!” and that conflicts with your microcovid budget, I think it’d make sense to re-examine your microcovid budget rather than rejecting the conclusion about driving.
Using numbers from this post: https://www.lesswrong.com/posts/ucjfY46L6qyXefvBT/quick-examination-of-miles-per-micromort-for-us-drivers-with
Seems roughly on par with the estimate I used of 1⁄4 the baseline risk of death.
[Question] Is driving worth the risk?
Great point. And this matches my experience as a Long Islander who was “only an hour” away from the city. When someone proposes, “Hey, we should go into the city!”, I recall it being met with hesitation more proportional to a hectic 2-2.5 hour trip than an easy hour long ride.
If the nearness to the city is a big factor for MIRI, and it sounds like it is, I think it’d make sense to get more data points on this subjective feeling of how big the hassle is. As well as data points on how long the trip actually takes, because adding up subway + walking + whatever seems very prone to the planning fallacy.
Plus, you are limited to making the trip while trains are running (so, no late-night hangouts then sleeping in your own bed).
I know that for the Long Island RailRoad, it still runs late at night, eg. 3am. It’s just that the frequency of departures goes down, eg. to once an hour. I’ve done the come home at 3am thing a bunch of times.
Although, I think it trades off with other potential goals, for example: community expansion or access to power. Those of you in miri know better where you are on timeline but research institute in the woods feels like it optimizes for a very particular move and may leave you less flexibility if the game isn’t in the state you imagine. As noted, I lack the information to know whether that’s a good or bad bet.
Great point. I too lack the information to really say, but I would imagine that the endgame would be to ~100x the size of MIRI, and that when you’re at that endgame you would want to be in a place where you can attract the top talent. Which I’ve always presumed to be SF or NY, but perhaps not in MIRI’s case. MIRI is looking for top AI researchers, and maybe that sort of person, for whatever reason, doesn’t prefer a tech hub.
Anyway, I’m thinking that if 100x is the endgame, maybe it’d make sense to optimize for that right now, since moving from one location to another is tough to do, especially once MIRI grows even more. Then again, maybe not. Maybe moving isn’t as difficult as I’m assuming. Or maybe it would make it too hard to reach the 100x point in the first place being in a tech hub, although when I write that out it doesn’t sound correct.
The US is such a car-centric place, that it’s tough. Europe would obviously be much better for that.
Yeah. That hit me when I was watching those driving tour videos. They say that places like Portland are really bike friendly. But looking at the Portland video next to the Amsterdam one, there was no comparison. The streets were filled with bikes in Amsterdam, and there were actual bike lanes everywhere there.
I am wondering what other areas may be most walkable, livable without a car.
From what I could tell, the issue is that the places that are truly walkable and livable without a car are all major cities, and those major cities are all super expensive to live in. Eg. NY and Boston. If anyone knows otherwise please let me know!
Ah, good point.
I just looked into injuries some more. The CDC says that for every 1 person killed, 8 are hospitalized and 100 are treated and released from the emergency department. Call it 1:10:100.
In the treated and released case, let’s say the cost of the injury is $1k. And let’s normalize things with death costing $1B. Since this injury is 100x more likely than death, we can say it costs $100k. Even if it were 1000k more likely than death it’d only cost $1M next to the $1B, so it seems like we can call this negligible and forget about it.
In the hospitalized case, suppose it causes issues lasting 50 years and costs you $100k/year (in utility). That’s $5M. Because it’s 10x more likely than death, call it $50M. But that’s still a good amount smaller than the $1B. And it was with conservative assumption of every person who is hospitalized ends up with a lifelong injury that causes significant distress. In reality it might be closer to $1M than $50M. So relative to death it also seems negligible. And perhaps more importantly, it doesn’t seem like it’s enough to outweigh the convenience of driving.