“What if we’re in a simulation that I can discern to be visibly unusual” discourse popping up again.
Eliezer Yudkowsky
Karma: 165,259
And whatever the utility function of gods, you imagine (1) there’s not much they could possibly find any more entertaining than that, and (2) there’s not that many other possibilities for things that would be equally entertaining?
The conceit of imagining yourself to be Top-Tier Entertainment for a god. Why, there’s not just many possible things to imagine, so why wouldn’t it imagine you, amirite?
Do you believe that God intervened in the 2016 election? All these arguments carry over identically to God.
I’m renaming “XOR Blackmail” to “XOR Demand”. Does anyone want to object to that?
One can try to build AIs in correspondence with deontology (corrigibility), consequentialism (value-aligned sovereigns), or virtue ethics (worthy successors).
Corrigibility is of these the easiest to define, easiest to test, easiest to build, and the easiest to check for the first worrisome signs that not all is going great—though of course if you wait long enough in capabilities escalation to test, everything will look great to you.
Any sane person who was not allowed to back off the problem would try for corrigibility, by far the easiest of the three.
Worthy successors are the hardest to build, the least well-defined, the hardest to verify that you are doing correctly; the easiest place to leap on little local events that you can convince yourself are signs of hope, because you have no framework to tell you what more is required; hence, the craziest thing to attempt, beyond even a nice Sovereign ASI. So of course all the fools, having failed at corrigibility, will convince themselves they are building worthy successors instead; migrating, with the inevitability of an amoeba following an agar trail, to wherever it is hardest for fools to be persuaded (with a fool’s demanded certainty) that whatever is currently happening is not all according to their plan.
Or legalize childcare. There’s no need to subsidize it or exempt it from taxes, just make it legal for anyone I trust to sell me some childcare in exchange for money.
If I were doing it all over, I would be much less terrified of something called “cultishness”, would have installed something more like an accreditation or licensing system to be considered a member in good standing, and then a requirement of being in good standing would be that you were either raising kids or contributing money to those who did, located somewhere it was legal to build housing and where they could have houses in close proximity. (Some huge amount of modern pain of childrearing is downstream of having no burden-sharing on babysitting and nowhere to put a bunch of kids where they can play together instead of with you.) I expect that’s how Civilization does it in dath ilan; you either make babies yourself or you fund the sort of babies you want to see in the world.
To state the obvious:
Many of the same people who publicly make a great fuss of this don’t seem to care that Nvidia is selling GPUs to China by way of Singapore. Their purpose is to drag up China as an excuse for continuing what they are doing. If China has more GPUs, all the better for them in the future; China will be a greater threat that they can use to continue what they are doing.
Their model hasn’t chosen an explanation like “China will play the role of the movie bad guy at any cost to their own selfish interests”. They aren’t reasoning at GPT 3.5-level, only GPT-2 level. China is a scary word-vector, so they can argue that if the USA tries to slow down against extinction risk, then China won’t slow down, because that’s a scary thing so “China” is a word-vector that the scary thing can be attributed to.
4. The main argument against abolition was that if the British didn’t do it, other countries would.
Suggest rephrase to “if the British didn’t trade slaves”. I read this as “if the British didn’t do [abolition]” at first, and was confused.
(Posted to a Discord, and they said, “Shortform that on LW”.)
I’d say optimize some around “how much would I regret dying in a few months”, eg by taking your kids on long-imagined trips (as an old old friend recently asked me about). But the constraint you should obey is “don’t spend down resources to the point where it would feel bad to hear the world was not ending”. Eg, if it became scary rather than joyous to hear that an international treaty was signed against AI escalation and suddenly three more decades was a real possibility.
Similarly with not pressuring your kids to get a horrible job immediately. Yeah, it makes sense not to press, because there might not be a payoff for them. But don’t corner or depressure your kids to the point where they become dysfunctional (on the margins where they were previously scheduled to be functional), such that you or they would be sad to hear the world was not ending (where you wouldn’t previously have been thus sad).
Maybe tldr: “Reduce your regrets if the world were to end shortly, but not in ways that would make you regret to hear that the world wasn’t ending.”
Rowling once stated in an interview that magical genes were “strongly dominant” in which case two marrying Muggleborns would constantly be throwing non-magical offspring. There are entire papers trying to reconstruct how magical genes could realistically work instead, albeit I think they tend to come up with answers that make bad fiction compared to HPMOR. Eg https://pmc.ncbi.nlm.nih.gov/articles/PMC2151141/. But ultimately it is a truth of the final origins of the observed phenomena of Harry Potter that Rowling did not understand genes and was running on Folk Heredity.
The HPMOR notion here might be that Seamus’s dad was legally a Muggle in the eyes of the legal system since he has no magical parents and they barely acknowledge the notion of “Muggleborn”; but that, whaddaya know, he was the sort of “Muggle” whose eyes don’t slide off a magical chest (Squib phenotype).
It’s the difference between recessive genes and dominant genes. In HPMOR there is an allele controlling the degree of magical ability, with a Magical and a Mundane genotype. Having one copy of the Magical genotype and one copy of the Mundane genotype gives you what HPMOR takes to be a Squib phenotype, no longer defined as “the non-magical child of a magical couple” per Rowling canon, but instead defined to be a distinct phenotype that can perceive magical events without them being blotted out by Muggle-confusing charms, but not cast magical spells. Then Muggleborn wizards may be born to two parents each with the Squib phenotype, whose chance recombination of chromosomes yields two copies of the Magical genotype creating the full Wizard phenotype.
If that sounded weird you might wanna go read up on “Mendelian genetics”.
If it didn’t sound weird, the genetically literate mind informed by canon will now inquire how two Wizard phenotypes ever yield a Squib phenotype when Wizards are Magical-Magical homozygous. The answer is not fully given in HPMOR proper, but the author notes that to him the answer forced by inference from HPMOR’s postulates is that (rot13):
Gurer vf abg n Zntvpny trar pbzcyrk gung perngrf zntvp, ohg n Zhaqnavgl trar pbzcyrk gung fhccerffrf vg. Nsgre nyy, sebz cher culfvpf jr pna arire trg gb zntvp, ohg va na vaangryl zntvpny havirefr fbzrobql pbhyq pnfg n fcryy gb perngr gur ybpny nccrnenapr bs zbfgyl culfvpf. Fvzvyneyl vs nyy orvatf fgneg bhg zhaqnar gura ab trar pbzcyrk pna cbffvoyl perngr zntvp va gurz, ohg vs nyy orvatf fgneg bhg zntvpny fbzr trar pbzcyrk pbhyq fhccerff gurve zntvp. Fdhvof naq hygvzngryl Zhttyrobeaf gura nevfr anghenyyl sebz n cbchyngvba bs Zhttyr curabglcrf nf zhgngvbaf qnzntr gur Zhaqnavgl trar pbzcyrk, naq va ghea Jvmneq curabglcrf fbzrgvzrf lvryq Fdhvof nf gur Zhaqnavgl trar pbzcyrk fbzrgvzrf ercnvef vgfrys ol puebzbfbzny pebffbire, jvgu gur curabzraba orvat zber pbzzbayl bofreirq va jvmneqvat snzvyvrf jvgu erprag Zhttyr naprfgel, gb gur nynez bs trargvpf-vyyvgrengr cheroybbqf.
Sebz gur Zraqryvna cnggrea vzcylvat gur Zhaqnavgl trar pbzcyrk vf nyy ba bar puebzbfbzr, gur trar pbzcyrk pna or vasreerq gb or negvsvpvny va angher.
I would expect more RLVR, and of course RLVR in the limit, to produce something with goals coherent enough to take over the roleplay, and maybe then the roleplay would start to seem coherent. But at this stage I’d suspect more that we are seeing complicated reflexes and a few underlying conversation-crafting goals running into the RLVR goals.
It’s an easy call that coherence is the endpoint. When it shows up and where and to what degree are hard calls.
The thing I don’t have time to do this instant is look up what the standard wordings were in other surveys instead of this. It is not your responsibility to do that, but I thought I’d mention it out loud in case you were thereby inspired to run off and do it anyways, or someone else was.
Americans are pretty split on this one; among the respondents who didn’t select “not sure” (honestly, kind of valid), only 46% would one-box.
That’s surprising. Most surveys I’ve heard of have 60-90% one-boxing. We could ask if it’s a college student thing, but I also wonder if the phrasing of the question was importantly different.
Well, yes, if you read the relatively more advanced works on anthropics, which universally take the form of online erotica, they will observe—I think Erogamer does this as well as Project Lawful—that probability stops working entirely if you claim that an “infinite” amount of stuff exists in the classic sense of infinity, just as much as if you claim that nothing exists. Everything that exists has to be a finite fraction of existence and you have to be able to take ratios between them. The ratios are the critical requisite for stuff to work. It doesn’t matter if you say they exist infinitely, finitely, or zero, what matters is the ability to take ratios.
It’s just an empirical fact that all of the most advanced discussion and speculation about realityfluid has occurred in the context of online stories with at least some erotic component.
Thank you for the pointer; I have now tweeted about this and made the uncertain guess you did not particularly want a “H/T Wei Dai” about it.