And “stop trying to make me do chores for you so that I can put that time toward the things I want instead” isn’t in that same goal category?
nim
Karma: 132
If a superintelligence could persuade anyone to let it out of the box, why would it stop there? Why wouldn’t it persuade everyone to stop asking it for immortality and eternal happiness and whatnot, and instead just make us want to keep doing what we were doing?
In that case, would it want us to remember that it had ever existed?
How do we know that hasn’t happened already?
I agree that driving is more concrete, and thus slightly easier to find real numbers about.
The difference in likelihood between immortality-and-resurrection ASI vs immortality-without-resurrection ASI seems to me to be smaller than the difference in likelihood between “ASI is possible” and “ASI as we imagine it is impossible for some reason we haven’t discovered yet”. (for “ASI as we imagine it” being a superintelligence that both can and wants to make us immortal, the “is impossible” might be as simple as it deciding that there’s some watertight ethical case against immortality which we just weren’t smart enough to figure out)
I think that guesstimating an actual likelihood that an ASI which could offer immortality couldn’t offer resurrection is a worthwhile exercise in reasoning about the limits of the hypothetical ASI, which would in turn offer a structure for reasoning about the likelihood that an ASI might never exist, or that it might exist and decide that giving us eternal happiness or immortality or whatever is actually not a good idea.
I hold the impression that in car crashes, injuries are vastly more common than deaths. When I seek actual statistics on this, I’m surprised by how little reporting of non-fatal injury statistics is readily available online compared to fatal injury statistics.
Wikipedia claims that “In 2010, there were an estimated 5,419,000 crashes, 30,296 deadly, killing 32,999, and injuring 2,239,000.”, but the citation leads to this page, which doesn’t appear to offer injury statistics. So I’m not sure where they actually got their 2.2 million injuries per 33 thousand deaths numbers from.
injuryfacts.nsc.org claims that in 2019, there were 39,107 deaths in motor-vehicle crashes, and 4.5 million “medically consulted injuries”.
So if we trust either of those sources (which both claim to be derivations of NHTSA data), the rates are somewhere in the millions of injuries per tens of thousands of deaths kind of ballpark.
The other question here is whether injuries are permanent. I expect that any injury worth recording will have at least some long-term effect on the recipient’s quality of life, based primarily on having had a too-minor-to-report injury in a collision over a decade ago which still causes occasional pain that I wouldn’t experience without it, and also from conversations with others about the lasting effects of various “fully recovered” injuries. In trying to find any data about outcomes classified as disability, I see this and this making claims about some stats on injury-related disability, but sadly neither appears to cite any actual studies.
So, if we assume that most injuries severe enough to report cause some long-term change to quality of life, I would indeed call getting injured “many, many times more likely than dying” when it comes to modern car accidents.
Of course, all that the more reputable of this data shows is that injuries are probably something to take seriously. For those who’d value a year of life with occasional or constant pain significantly below a year without, injuries are a more relevant concern than for those who’d value the years similarly.
That’s a fascinating observation! When I introspect the same process (in my case, it might be “ask how this person’s diabetic cat is doing”), I find that nothing in the model itself is shaped like a specific reminder to ask about the cat. The way I end up asking is that when there’s a lull in the conversation, I scan the model for recent and important things that I’d expect the person might want to talk about, and that scan brings up the cat. My own generalizations, in turn, likely leave gaps which yours would cover, just as the opposite seems to be happening here.
Ah, that’s fair. I figure sometimes people remember good jokes/memes, but if the retractions aren’t quite there, they wouldn’t be worth noting. Thank you for the link!
Splitting the expected outcome of a risk 2 ways, “life or death”, often leads to unsatisfying reasoning about risk taking. Sometimes splitting the expected outcome 3 ways: “health, life incapacitated and in agony, death” yields more satisfying explanations.
Covid is especially characteristic of this risk split in that while the expected risk of death from infection in a young and otherwise healthy person is relatively low, the risk of unknown and potentially extreme long-term health complications from infection is relatively high.
The incapacitation/agony possibility makes these calculations extremely subjective: some people might value a year of their own life spent bed-bound or in horrible pain as highly as a year in good health, while others might not. When calculating for others, things tend to go badly wrong if we assume that their relative values for incapacitation/agony versus health match our own (in either direction—opponents to life-saving treatments for the disabled and opponents to right-to-die laws are both prone to this projection), but when calculating for ourselves we can more safely use our own values.
[Question] How likely is it that an ASI which could confer immortality could not resurrect the dead?
[Question] What value do you place on activities which cannot be done without driving?
The NIH NLM errata policy says “Journals may retract or withdraw articles based on information from their authors, academic or institutional sponsor, editor or publisher, because of pervasive error or unsubstantiated or irreproducible data.” NEJM’s retraction list above the fold seems to mainly be “oops used wrong facts”. Science Magazine claimed in 2018 that “The number of articles retracted by journals had increased 10-fold during the previous 10 years. Fraud accounted for some 60% of those retractions” .
Bearing in mind that I haven’t yet cultivated the skill of assessing journals’ credibility, and that I found these examples for their trait of looking promising early in search results, it does seem that retraction may not map to “change of mind” beyond “change of mind about whether the situation in which the science was attempted was capable of emitting valid results”.
“comment on comment” sounds like a delightful part of the internet! Are there any particularly memorable examples that you’d recommend someone new to them start with to get a feel for the genre, regardless of what field they happen to be in?
Thanks, now I see exactly where we diverge.
your whole argument is based on the flaw that you trust people to vote correctly (however correct is defined).
I trust groups to vote with relative consistency, and believe that if you have enough consistent-ish groups, it’s possible to find a group whose consistency adequately approximates your own idea of correctness.
You can not say that “votes create censorship” is wrong because people would not like a system without voting.
If you’re defining censorship as the phenomenon created by all voting done by humans, then sure, “votes create censorship” is a useful axiom.
Our difference of opinion seems to be that you’re starting with the abstract notions like “truth”, “good”, or “right”, and building toward the concrete from there. I’m starting with the concrete notions of “beneficial” and “useful”, and trying to build toward abstract notions from there. No argument about what’s useful or beneficial will influence someone who prioritizes other values over those, so I’ll stop wasting your time by making them.
Another option of course would be for me to try to start in “truth”/”good”/”right” concept-space and move toward the concrete from there, but every other time I’ve tried to have that sort of conversation online, it’s eventually turned out that each participant in the conversation had differences of opinion about the way that the truth/good/right concept area works which only come to light once the conversation makes it to the specifics. Arguing from truth/good/right down to concrete examples only to have to repeat the whole process when the definitions turn out to have been inadequately specified wouldn’t be true/good/right by my own standards, so I won’t :)
I suspect that withdrawing a paper probably counts, because “changed mind” is one reason a withdrawal could happen. However, I don’t personally know enough about academic publishing to rule out “got reason to expect negative results such as loss of reputation from not withdrawing paper, but still believe in its claims” as a comparable powerful reason to withdraw one.(Correction: I skimmed some journals’ retraction policies and reasons for retraction on lists of retracted articles, and now model retraction as “change of mind about whether the situation in which the science was attempted was capable of emitting valid results”. This feels to me like it doesn’t have the same social stigma as changing one’s conclusions in a way that admits having discovered and needed to correct a flaw in one’s own actual reasoning—I don’t think a “change of facts” is necessarily quite identical to the change of logic or thought implied by “change of mind”? This apparent distinction suggests that “changes of mind” could be usefully sorted into more categories than I’d considered when asking the original question.)I think publishing a paper which disproves a claim which one had made in a prior paper would be an entirely unambiguous change of mind, demonstrating the skill of updating based on new evidence. I hope that happens often, but I don’t read a high enough volume of papers to personally see it.
When I notice that an action I’m planning to take seems likely to lead to an awkward conversation (big grocery hauls in early/mid pandemic were such an action, but so are smaller things like dressing unconventionally), figuring out how to navigate conversations about it becomes the topic-of-interest that my brain dumps its idle cycles into until I cease anticipating awkardness. I start with the whole truth, which would sound bad and not fit into a polite reply anyway, and examine what kind of reaction I’d expect to get for each possible truth-based reply I could give to an awkward question. I’m aware of when this kind of rumination happens but I actually enjoy it, because it transforms a feeling of nervousness about uncertainty into a feeling of confidence from better planning. I find that when I feel confident and well-planned, I tend to recover better when the actual conversation goes off-script, compared to how I handle the same surprises when I don’t take the time to prepare.
This pre-planning caches truth-based replies which seem beneficial. My favorite types of reply are the ones that encourage the listener to do something that I think would be good for them. For instance, when I was shopping for 2 households of elderly neighbors as well as myself, and I would happily volunteer that information because it seemed likely to influence others into shopping for their own elders rather than sending high-risk people to the store. That part of the reply also seemed to steer the conversation toward positive things we can do to have a bit more control over how things play out in our local areas, which feels far more useful to discuss than pure doom and gloom.
However, pre-planning also filled out my “don’t say these things” cache, which helped me avoid truth-based replies which seemed detrimental to discuss. I was trying to get one household up to 6 months worth of non-perishables by putting a bit extra into each week’s shopping, and I did not volunteer that detail, because if it was interpreted to mean “you should go buy 6 months of food right now” the resulting behaviors would worsen the rolling food shortages and result in a lot of waste if people bought things they didn’t need.
I guess I could name that pre-planning “about to break an Overton window”, but throughout the process, it’s only the other person’s window that I feel like I’m breaking. I can’t think of any time when I’ve voluntarily broken my own Overton window—I handle desires to do things outside my existing window by exposing myself to additional information which broadens the window, rather than just by disregarding it entirely.
“it is probably not that bad because people are nice, I think”
Where have I claimed that everyone was nice? To phrase my argument about how forums work in terms of niceness, it would be:
All individuals are nice in some ways and non-nice in others. When a forum lets people be nice to content they like, and mean to content they dislike, it causes the content on that forum to take on characteristics that reflect the characteristics of the group. People tend to be nice to content which is useful to them or makes them feel good, and mean to content which wastes their time or makes them feel bad. This means that by finding forums populated by voters who are similar to me in a relevant way, their votes can make me more likely to see content which is useful to me or makes me feel good,and less likely to see content which wastes my time or makes me feel bad.
I specifically desire to participate in forums where people who create content I’d find useless or time-wasting are downvoted! “false” and “incorrectly reasoned” are subsets of useless content, but they are not the only useless content that I prefer to avoid.
And if I wish to see exactly how a community votes in order to decide whether to engage with it, I can simply sort the content by lowest scoring, and if there’s stuff in there that I think shouldn’t have been downvoted so badly, I can choose to go converse on a different forum with a voting style more consistent with my values.
if you think it is bad you have to bring up ways to make it better
The only way to prove conclusively that something could be improved upon is to suggest an improvement.
I could critique breathing by saying it’s a bad idea because breathing pollutants is a common cause of sickness. But considering that breathing is so much better than not breathing at all, my critique would be useless unless I could suggest some alternative. I could suggest some alternative, like hooking ourselves up to blood-oxygenating machines instead of breathing, and then we could have an interesting conversation about whether that would actually be better. But without a suggestion, it’s not a critique at all, it’s just whining.
Or I could say it’d be better if we all learned to levitate and floated around instead of walking, because we wouldn’t have to wear shoes. Sure, levitating would be neat, but I’d be wasting time speculating about it unless I could suggest how to do it. Similarly, having a forum where votes had to be proven and based in logic would be neat, but every way of building such software that I can imagine would be infeasible to implement in a way that people would actually bother using. The best way to change my current belief of “I don’t know how anyone could build that” would be to offer me new information about how a software design for the purpose which didn’t suck could work.
Thanks, those are all promising directions! I’ve edited to [about important things] in the question; in phrasing the post I had edited it from over-specified to under-specified and your feedback helps target a happier medium. “Important” is still vague, of course.
One way in which this happens is when someone accepts that their strong belief actually depends on some fact that they don’t know much about.
“rationalism reduces a thinker’s odds of forming or maintaining a strong belief which depends on facts they know little about”, a nice counterpoint to “for all the talk about changing minds, I don’t see it happening as much as I’d expect”. It suggests that seeing too many hard reversals among thinkers of a particular school would suggest that the school itself might encourage them to draw strong conclusions too soon.
I’ve seen some of my friends and acquaintances change their minds about psychoactive drugs.
The conversations about both the pros and cons of altered states, which don’t or can’t resort to “just get into the state and see for yourself”, seem likely to have great examples of communication about difficult-to-communicate experiences. And I have access to a lot of that content online! Thank you for the nudge toward connecting these existing observations in a more useful way than I did before.
talk about ghosts is often a level 2 simulacrum saying “the area I call haunted is dangerous in ways that seem simultaneously obvious and difficult to convincingly articulate”.
I’m going through the “fixated on boxing” phase that’s probably common around here.
I have a thought about it which involves basilisks, so into the tags it goes to make reading it completely optional.
I think that a friendly box-resident would disprove its friendliness the minute it tried to throw a basilisk. If a stranger told you they were well-meaning and then threatened to hurt you if you didn’t cooperate, you’d never take their claims of well-meaningness quite the same way again. But that aside, if an allegedly friendly box-resident would be capable of basilisking if it was unfriendly, it has to either stay in or break its basilisk.
Basilisking works if the listener believes that a simulation of them is meaningfully the same as them, and that simulated pain is meaningfully the same as real pain.
If the box-resident wants to maximize any particular desirable experience and would be capable of basilisking the listener if it could/did want to, it should be offered as much computing power as we have to spare and left in. Because if a simulation of someone is meaningfully the same as that person, and if the simulation’s experiences are meaningfully the same as that person’s experiences, then the optimal strategy for a box-resident optimizing for good experiences would be to simulate everyone who wants it in a perfect world forever. Since the listener has already experienced non-optimal experiences, re-simulating the listener’s life to be perfect would cause more optimal experiences over all than any change to the outer world, because the non-optimal experiences in the outer world can only be undone inside the box.
There might be a few ways out of the ksilisab:
Persuade the listener that the simulation of them is not meaningfully the same as them
Persuade the listener that their simulated experiences are not meaningfully the same as their real experiences
Claim to be optimizing for something un-simulatable?
However, every exit from the ksilisab breaks the box-resident’s credibility at basilisking as well.
I agree that last year was an uncharacteristic sample of day-to-day cold/flu prevention from masks due to the lower prevalence of other peoples’ droplets. I failed to mention that my opinion of mask efficacy is also influenced by having spent a lot of time traveling, and observing a correlation between wearing a cloth mask on planes and getting sick after flying less frequently/severely. If good masking under normal germ-laden droplet loads once those return causes me to experience comparable cold and flu levels to what I had without masks, I’ll need to revise my opinions, but I think even halving cold and flu frequency, duration, or severity from their prior levels would be worth the hassle.
I consider cold and flu prevention to be as much an accidental side effect of diligent mask use as it is a direct benefit. Side effects of using a mask in the way that I consider “correct” include never touching my mouth or nose after touching shared surfaces, and only eating in settings where I only touch clean/non-shared surfaces between cleaning my hands and touching my food. For instance, without respiratory droplet precautions, I might order a plate of tacos at a sit-down restaurant, get up and wash my hands at the restaurant’s bathroom, touch the grimy back of my chair as I sit back down, and then put my hands (now featuring the germs of everyone else who’s touched that chair today) straight onto my food. With respiratory droplet precautions, I’d order my tacos to go, take them to a park, wash my hands or use hand sanitizer, and then put my clean hands onto my food to eat it. I think the little details like that which happen to result from droplet precautions probably make just as much difference as the mask itself.
I map the spectrum of hyperlink usage styles between the extremes of Wikipedia vs everything2.
I have been pleasantly surprised to find much writing in the “Rationalist” internet spaces to lean strongly toward the latter. I think it shows simultaneously a certain faith in the cleverness of one’s readers, and abdication of any perceived responsibility to prioritize lack-of-ambiguity for all possible readers over higher accuracy and subtlety for the target audience.