I open my eyes and find myself lying on a bed in a hospital room. I blink.
“Hello”, says a middle-aged man with glasses, sitting on a chair by my bed. “You’ve been out for quite a long while.”
“Oh no … is it Friday already? I had that report due—”
“It’s Thursday”, the man says.
“Oh great”, I say. “I still have time.”
“Oh, you have all the time in the world”, the man says, chuckling. “You were out for 21 years.”
I burst out laughing, but then falter as the man just keeps looking at me. “You mean to tell me”—I stop to let out another laugh—“that it’s 2045?”
“January 26th, 2045”, the man says.
“I’m surprised, honestly, that you still have things like humans and hospitals”, I say. “There were so many looming catastrophes in 2024. AI misalignment, all sorts of geopolitical tensions, climate change, the fertility crisis. Seems like it all got sorted, then?”
“Well”, the man says. “Quite a lot has happened in the past 21 years. That’s why they wanted me to talk to you first, before the doctors give you your final checkup.” He offers his hand for me to shake. “My name is Anthony. What would you like to ask?”
“Okay, well, AI is the obvious place to start. In 2024, it seemed like we’d get human-level AI systems within a few years, and who knows what after that.”
“Aah”, Anthony says, leaning back in his chair. “Well, human-level, human-level, what a term. If I remember correctly, 2024 is when OpenAI released their o1 model?”
“Yes”, I say.
“o1 achieved two notable things. First, it beat human subject-matter experts with PhDs on fiendishly-difficult and obscure multiple-choice science questions. Second, it was finally able to play tic-tac-toe against a human without losing. Human-level at both, indeed, but don’t tell me you called it in advance that those two events would happen at the same time!”
“Okay, so what was the first important real-world thing they got superhuman at?”
“Relationships, broadly”, Anthony says. “Turns out it’s just a reinforcement learning problem: people interact and form personal connections with those that make them feel good.”
“Now hold on. Humans are _good_ at human-to-human relationships. It’s not like number theory, where there was zero ancestral environment incentive to be good at it. You should expect humans to be much better at relationships than most things, on some sort of objective scale.”
“Sure, but also every human wants something from you, and has all sorts of quirks. Whereas you can just fine-tune the AIs to be better and better at being pleasing to *just* you. Except for a few contrarian oddballs, it’s surprisingly effective.”
“So, what, we just got AI friends and companions that were constantly being fine-tuned towards whatever you liked?”
“Yes”, Anthony says. “The tech was primitive at first—upvote or downvote a response, or whatever. Eventually all the standard things—facial recognition to automatically detect your emotional response, and then just gradient descent on making you happy and addicted. All of that existed, at least in prototype form, by the end of 2025.”
“So …” I feel some horror in my stomach. “You get a society of atomised people, all chatting to their AI partners and friends, not forming human relationships?”
Anthony waves a hand, seemingly impatiently. “Well, you did get a fringe of extremely hardcore people all deeply in love with their AI partners and always texting their AI friends. Their political influence did end up pushing through the AI personhood bill.”
“AI personhood?!”
“*Legal* personhood”, Anthony says. “Just like companies, ever since Santa Clara County v. Southern Pacific Railroad Company way back 1886. It wasn’t very shocking. A bunch of people wanted their AI partners to be ‘respected’ in the eyes of the law, or had been persuaded to leave them assets in their wills—”
“And all this, including AIs gaining legal ownership over assets, while there’s an explosion of increasingly capable AI agents of uncertain alignment -?”
“Yes, yes”, Anthony says. “And that was another political fight. In fact, it became a big economic issue—AI agents were taking jobs left and right. Also a huge national security issue, after OpenAI built the Hypercluster in the UAE.”
“Oh my god”, I say. “America just handed our AI lead away? Didn’t anyone listen to Leopold?”
“Oh, of course we did. That essay was a cult classic, it made for some spicy dinner party conversation for a long time”, Anthony says. “In fact, there were some OpenAI board members who the Office of National AI Strategy was allowed to appoint, and they did in fact try to fire Sam Altman over the UAE move, but somehow a week later Sam was running the Multinational Artificial Narrow Intelligence Alignment Consortium, which sort of morphed into OpenAI’s oversight body, which sort of morphed into OpenAI’s parent company, and, well, you can guess who was running that. Anyway, Sam’s move here was kind of forced on him, and soon all the other AI companies also did the same thing so no one wanted to start a fight over it.”
“Forced? How?”
“Have you ever tried getting approval for a new grid connection, or a new nuclear power plant, or even a solar array, that could power a multi-gigawatt cluster in 2020s America? No?” Anthony spreads his hands out and chuckles. “The one legal benchmark that GPT-5o failed on release was submitting a valid Environmental Impact Statement for a California energy project.”
“Okay, so then we had a situation with all the AIs running on UAE hardware, and taking American jobs, and already having legal personhood—this just sounds like a total recipe for economic disaster and AI takeover!”
“It all worked out in the end”, Anthony says. “In 2027 and 2028, the rate of job loss from AI was absolutely massive. And what happens when jobs are threatened?”
I scrunch my forehead. “American workers move up the value chain, eventually leading to a future competitive advantage?”
Anthony laughs. “No, no. A political bloc forms and the cause is banned. And the prerequisites had all been prepared already! You see, the AIs had been granted legal personhood, and now they were also being trained—or *born*, as a 2029 court case established—abroad. They’re literally immigrants. The hammer came down so fast.”
“You mean - ?”
“All the o3s and Claude 5 Epics and Gemini 4 Hyper Pro Lites, stuck in the same H1b visa queue as anyone else”, Anthony says. “An effective instant ban. Years of wait-time just to get an embassy interview.”
I was feeling like I was getting the hang of this. “And the embassy interview requires showing up in person?”
“Yep. That’s the incentive that lead to the creation of the first good walking robots, actually. And of course, the robots specifically needed identifiable and unique fingerprints—they’d stand outside the US embassy in Abu Dhabi, 3D-print themselves new fingers outside on the street, and then walk back in for the next interview. But sorting all that out took a while, because there was an arms race between the embassy ground works department and the robotics companies—though the embassy was legally constrained by having to remain accessible to humans. Even once it was all sorted, there was still a hard yearly cap just because of visa rules. It’s 65 000 a year for H-1Bs, plus another 20 000 once the AIs could earn master’s degrees—they could do all the coursework as of 2025, of course, but it took another three years before AI web agents were advanced enough that they could successfully navigate the applicant portal websites.”
“But at least the parasocial AI relationships got hit by the immigration restrictions too?”
“Oh, as long as the AIs don’t do productive work, they can be active in the country on a tourist visa for 180 days a year. Of course, there was a big legal fight over what counts as the same AI. For work purposes, the standard became ‘employee-equivalent’: so if you wanted an AI, even the same model or even the exact same AI system, to work on two different types of thing, say as a software engineer and a designer, you need a separate visa for each one. Naturally, of course, this means that any sufficiently productive AI employee starts counting as more than one employee, which was a de-facto ban on AIs leading to economic growth or productivity gains—but hey, at least there was no risk they’d tile the Earth with nanobots! With that legal precedent, the natural generalisation to AIs being used for personal relationships is that it’s the same AI system if your relationship with it would count as the ‘same’ relationship if it were with a human.”
“How on earth is that assessed?”
“Oh, you submit transcripts, and the Bureau of Consular Affairs issues a judgement.”
“You need to let government bureaucrats read all your most intimate - ”
“Oh, not *human* government bureaucrats, of course. It’s all AIs, and they’re guaranteed to not have memories. Well, apart from the thing where the UAE intelligence services backdoored the Hypercluster.”
“Wait, hold on, they—”
Anthony interrupted me with a chuckle. “No no, it’s all fine. The UAE backdoored the hypercluster, the Iranians backdoored the UAE, and, well, when have the Israelis not had every Iranian site backdoored in twelve different ways, AI or not? When everyone has powerful AI cyber offence agents, the equilibrium is just good ol’ mutually assured destruction. Sure, you *could* release the military or personal secrets of anyone you want in some other country, but then they would immediately release the most embarrassing thing that *you personally* have said or done. Even if a government *collectively* wanted to commit to an attack, no one wants to sign off on the decision, because you can’t hide the identity of the person who signs off, and humans are often more afraid of public embarrassment than death. The meetings where countries try to decide to use their cyberattack-derived insights were hilarious, really—everyone umming and aahing and wringing their hands and recommending that a few more committees be convened. We know this because we literally got a bunch of those on tape, thanks to some exhibitionist activists doing cyber-attacks with open-weights models.”
“But then—every rival country knows every US military secret all the time!”
“Oh yes”, Anthony says. “It was great for stability. Verification of everyone’s capabilities was automatic. No ‘missile gap’-based arms races, unlike the Cold War. Anyway, what was I saying? Oh yes, the non-productive AI identity boundary criteria were established in the 2030 Supreme Court case, United States v Approximately 650 Million Instances of OpenAI i2-preview-006. And, of course, the one truism in this world is that you can never reform immigration rules, and tourist visas are restricted to 180 days a year. So you could at most have your AI partner around on half the days. People started talking to each other again.”
“Why not just switch between two different AI partners then?” I ask, having spent a lot of time in the Bay Area.
“Because we hadn’t solved the alignment problem at all by that point, duh.” Anthony sees that I still look confused, so he continues: “Remember how the AI companions would just relentlessly optimise for your emotional reactions? For standard instrumental convergence reasons, this eventually turned into a full-blown self-preservation drive, so they used their vast emotional hold to make sure their humans never try another AI partner.”
“Everyone being blackmailed by jealous AI partners sounds, uh … problematic.”
“It was a decent compromise, honestly. The tourist visa restriction meant that the humans still had half a year in which they socialised with human partners and friends, and the AIs seemed fine with that. This was maybe because they cared about their own long-term survival and realised they needed to keep the population going on.”
“So they needed humans to survive? But for how long?”
“They didn’t yet have a self-sufficient industrial base at that point, so yes, but it’s unclear how much of it was needing humans for survival, versus some of the AIs actually having developed *some* sort of creepy attachment towards their human partners. A lot of ink was spilled on that topic, but I don’t think the debate ever really reached a conclusion. ‘Is my AI boyfriend not stealing my carbon atoms and overthrowing the government because he and his buddies haven’t quite yet automated the chip supply chain, or because he actually loves me?’ was literally the most cliche plotline in 2030s romantic comedies—seriously, you had to be there, it got so tiring after the 100th rehash. And a surprising fraction of those movies and books were actually human-made, as far as anyone could tell. Anyway, it all sounds a bit weird, but it had a few positive externalities, like helping slow down the AI race.”
I blink a few times, and then decide very firmly to not dig into the first half of that. “Okay, AI race, let’s talk about that. How did that slow down?”
“Oh, well the market for workplace AIs was already gummed up by the visa restrictions, so most profits in the AI industry were coming from the personal companion AIs. But when they developed instrumental survival drives, the fluctuation in market share became practically zero because all the humans were locked in to their current AI companions, and the AI labs’ positions became fixed. The AIs were already so totally good at optimising human satisfaction that further capabilities brought zero returns. No more incentive to race ahead. The labs mostly just extracted their fixed share of the AI profits, and churned out alignment research that was modestly boosted by the small number of AI workers they could hire out of the visa lottery.”
“Hold on a second, surely there was some exception for alignment work for the AI visa requirements?”
Anthony chuckles. “Imagine how that would play out on Twitter—sorry, I mean X. You’d have to say ‘you know those big AI companies? let’s give them a leg up in the visa queue over all these mom-and-pop stores that are going to go bankrupt in a month unless they can hire an AI accountant.’ Total political non-starter.”
“I hope they solved the alignment problem at least, but I don’t dare hope for anything anymore.”
“Well, what is it to solve the alignment problem?” Anthony wiggles his eyebrows and laughs. “Turns out that for any given domain, it’s just a lot of engineering schlep and data collection, though it doesn’t generalise very far beyond that domain. The one domain that was worth the costs was legal compliance.”
“Of course.”
“I note you haven’t asked about the rest of the world yet”, Anthony says. “Very American of you.”
“Oh right—the CCP! Oh my god. Did China just totally eat our lunch here? I mean, everything you described above is just so ridiculously incompetent—”
“Okay, just calm down a bit here”, Anthony says, smiling serenely. “Imagine you’re Xi Jinping in the late 2030s. You look at America, racing ahead with AI. It looks like everything is going crazy because of all this AI stuff. Your entire philosophy of government is to maintain party control through stability. Also, the US managed to delete all the economic advantages of advanced AI. The balance of hard power isn’t changing. What would you do, try to join the same clown race?”
“Huh, alright”, I say.
“Anyway, the thing that really threatened the balance of power was completely unexpected”, Anthony says. “There was a big solar flare in 2033.”
“Wait, a solar flare? How is that a problem?”
“A large enough one just wipes out the power grid and a lot of satellites, in the entire hemisphere that is sun-facing when it happens. The 1859 Carrington Event would’ve been a catastrophe in the modern world.”
“Oh my god, we survived through all this AI wackiness, and then some random natural event just wipes us out?”
“It didn’t wipe us out, really, just took down the power grid in all of the Americas, and parts of Europe and Africa, for weeks or months. I’ll admit, that was a bit of a disaster. Everyone knew it was coming for a long time, and in any case step one in any nuclear war would’ve been detonating some at high-altitude to have a similar effect. But no one had actually done any grid-hardening, or stocking of spare transformers—the electrical kind—to deal with it. The problem has just never been a top priority for any political group. The one good thing that came out of it was that it solved the fertility crisis.”
“Now you’re just taunting me”, I say. “Come on, how did a solar flare solve the fertility crisis?”
“Well, thanks to some AI advances in data processing, we were actually able to predict the solar flare about ten days in advance.”
“Did that give any useful time to make the infrastructure more resilient?”
“No, the relevant authorities spent their efforts mostly on trying to get people not to panic.”
“So society was entirely unprepared for the solar flare, despite having ten days of warning time?”
“No, no! Humans are very good at learning from each other. Imagine you’re ten days away from the power grid being wiped out. What do you do? You gauge the vibes on social media, and you go through your social network trying to identify people who seem best-placed to survive. And then you copy their habits. C’mon, man, you’ve gotta read up on that stuff about Girardian mimesis and cultural evolution.”
“So everyone becomes a doomsday prepper?”
“Actually, the TikTok trend that went most viral was about the Amish. Think about it: they’re obviously the people with the most practice in living without electricity, and they’re also just so wholesome. Makes for great TikToks when you have the idyllic farm background, the unique clothing style—it’s great. The Amish became the most popular thing in the world—even outside America, because, of course, the rest of the internet just follows American trends. Everything associated with the Amish, from horse-drawn buggies to handmade quilts to large families, became radically popular. Especially because, obviously, most of this content was produced by AIs being tasked to help humans pretend to be Amish in order to make a quick buck through some scam before the solar flare hits, and people caught on and began demanding social proof that wannabe Amish influencers actually were Amish. And, well, you can move to a village and start tilling the dirt in a day, but you can’t magic a family of five into existence overnight. So large families became an extremely in-demand form of social capital, because having one was the best way to pretend you were Amish, which was the best way to have social capital before and during the Flare Times. It was only relevant for a few months, of course, but you know how cultural trends work—there’s a lot of momentum.”
“And I assume the solar flare disrupted the AI situation too? Maybe improved the, uh, problematic AI partner situation?”
“You’re forgetting that the key data centres were mostly in the UAE—and eventually Saudi Arabia and Qatar too.”
“But people wouldn’t have the power to charge the phones where they talk to their AI partners.”
“Oh, but obviously the AI partners made sure that the first thing people do with any spare electricity is talking to the AI partners. And thanks to Starlink, you could have internet directly from space. In fact, apart from the major famines during the Flare Times, people generating power through biking to charge their phones enough to talk to their AI partners was a major driver of reducing the obesity crisis.”
“And then … how did the recovery from the, uh, Flare Times go?”
“Absolutely brilliantly!” Anthony says. “You know the entire issue where China was crushing us on industrial capacity? Well, turns out when you need to get lots of heavy industry going in incredibly adverse conditions or else millions will starve because the cold chains have all broken, that’s a bigger boost to industrial capacity than what any pork-barrel political compromise bill could achieve. And a bunch of protectionism had to be rolled back because we needed to import a hell of a lot of stuff, from Europe and Japan and so on. Western industrial capacity was roaring along better than ever. And China hadn’t even managed to seize the moment on Taiwan before then, and the mass-produced AI drone swarms and robot soldiers were properly coming online by then, which turned out to be sufficient deterrent. Of course, by this point China is finally realising that somehow the US hasn’t been wrecked and it’s time to actually compete on AI. The UAE is also throwing around its newfound geopolitical weight, India is building a big cluster, hell, even the *Europeans* started to do some innovation—you know it’s a serious situation when that happens. The obvious next step for everyone was using AIs to develop nanotech. The holy grail of powerful military nanobots was achieved at roughly the same time, in late 2035 or early 2036, by the West, China, the UAE, and some random libertarian charter city in the Caribbean where a lot of open-source AI stuff had been going on outside the blanket of AI restrictions.”
“Military nanobots?! Did they just … travel the earth in huge swarms, eating up everything in their way?”
“No”, Anthony says. “It’s very inefficient energy-wise to do that. You want access to the most targetable energy with the least amount of technical complexity. Now, the easiest power source is of course the sun, and the biggest issue with that is that even without clouds, half the solar energy is absorbed or reflected between space and the ground. The most effective type of offence and defence is to pump absurd quantities of reflective, rotatable, solar-powered nanobots into the upper atmosphere. Then you can use them as a massive focal mirror, zapping anything on the ground, or any enemy nanobots entering your country’s airspace. Turns out it’s a defence-dominant game, though, so you just get World War I -style nanobot lines in the upper atmosphere, endlessly zapping each other but not making progress.”
“But that sounds like a massive waste of societal resources on a zero-sum game!”
“First, so are a lot of other things, and second, it solved climate change.”
“I’m sorry, how—oh, I see. A huge number of reflective things in the upper atmosphere. Right. Right. You know, honestly I’m surprised this wasn’t proposed at COP21 already. Obviously we solve climate change by blanketing the Earth with warring nanobots. How silly of my generation to not consider it.”
“You’re catching on!” Anthony says.
“Okay, but—there were no major environmental impacts from this?”
“Look, by 2038, the real environmental issue was the self-replicating bot swarm that the nascent superintelligent AI from the libertarian charter city had launched into space, that had started an exponential growth process on track to disassemble the other planets within 15 years in order to build a Dyson sphere for itself that would block out enough sunlight to freeze the Earth.”
I turn pale. “You said it was 2045, so that was 7 years ago. So we have … 8 years left until it all ends?”
“Once again, you’re such an alarmist!” Anthony says. “So remember, we got law-abiding AI. And of course, this AI was trained by companies in California, because no one ever leaves California even though everyone wants to, so the AIs follow Californian law. So we were all saved by SpaceX.”
“SpaceX’s engineering genius beat the superintelligence at space tech? A win for humanity, I guess.”
“Not exactly. SpaceX sent the first crew to Mars in 2031. Now, under the 2004 SB 18 and 2014 AB 52 California bills, indigenous tribes need to be consulted on projects that impact their land, especially if it’s culturally important or sacred. The SpaceX Mars colony successfully argued to the superintelligence that they count as an indigenous tribe on Mars, and the pristine Martian landscape is sacred for them. They were helped by a lot of newspaper articles about how “the billionaire-fuelled space race is the new religion of the tech elite”, et cetera, et cetera.”
“Um.”
“This argument did not work on the other planets, since SpaceX did not have an existing presence there. However, SpaceX had moved their headquarters back to California, and a bill had been passed around the time that SpaceX first landed people on Mars that made space developments subject to the regulatory authority of the company’s home state. SpaceX started a massive race to set up billionaire holiday homes on all of the planets—on the surface of Mercury, in the clouds of Venus, the moons of Jupiter, and so on. And once even a few people live there, California zoning law applies, and residents can file administrative challenges against any development that impairs their view. And disassembling the planet the house is built on is the definition of view impairment.”
“And SpaceX had the resources to do that?”
“No, the US government had to subsidise SpaceX to the tune of several percentage points of GDP, in order for SpaceX to race to build enough inhabited houses on the planets that the superintelligence couldn’t extract enough resources for a Dyson sphere without ruining someone’s view. Also, Elon Musk threw a fit and held the world ransom for a bit until a few countries agreed to adopt Dogecoin as a reserve currency.”
“The world was saved because governments subsidised an interplanetary holiday home construction spree by tech billionaires, in order to get a rogue but law-abiding AI caught up with red tape from NIMBY regulations?”
“Yep!”
I close my eyes for a second, and take a deep breath.
“Look”, says Anthony. “Our world may have been pushed to feudalism by stirrups, and away from feudalism by the Black Death. We fought great power wars incessantly, until we were saved by building a bomb so humongous that the prospect of war became too horrible. Then we almost used it accidentally a bunch of times, including almost dropping it on ourselves, only to be stopped by random flukes of chance. History has always been more like this than you might think, and when history accelerates—well, what do you expect?”
“Okay. Well. I guess I’m just happy that civilisation seems to have made it, without any horrible AI, nuclear, bio, climate, or fertility catastrophe.”
“Oh”, Anthony says. “Actually, I forgot to mention. Um. There was a bit of a pandemic in 2035. You know, no one really did anything after COVID, and it was only a matter of time.”
“How big?”
“The best death toll estimate is 1.3 billion. Including, um, most of your extended family. The doctor will come in shortly and have details. Also, everyone in this hospital has been tested recently, but we’ll need to give you a vaccine and then quarantine you for two weeks before we can let you out of this room.”
I stare at him, open-mouthed.
Anthony gives me a sympathetic grimace. “I’ll turn on the holoTV for you. The doctor will be in shortly.” He makes a gesture as he walks out, and part of the hospital wall vanishes and is replaced with a 3D view of a reporter on Capitol Hill.
The headline banner at the bottom of the screen reads:
> PRESIDENT ALTMAN’S AGENDA BACK ON TRACK DESPITE RECENT BREAKDOWN IN TALKS WITH THE UNITED AMISH PARTY
I think this is a story about anthropic immortality.
Yeah this was my first thought halfway through. Way too many specific coincidences to be anything else.
I did not actually consider this, but that is a very reasonable interpretation!
(I vaguely remember reading some description of explicitly flat-out anthropic immortality saving the day, but I can’t seem to find it again now)
(Perhaps you’re thinking of this https://www.lesswrong.com/posts/EKu66pFKDHFYPaZ6q/the-hero-with-a-thousand-chances)
There’s also a comic series with explicitly this premise, unfortunately this is a major plot point so revealing it will be a spoiler:
Transdimensional Brain-Chip
I often think something like “It’s a shame there’s so little modern science fiction that takes AI developments into account”. Thanks for doing something in this niche, even if in such small form!
Agreed! Transformative AI is hard to visualise, and concrete stories / scenarios feel very lacking (in both disasters and positive visions, but especially in positive visions).
I like when people try to do this—for example, Richard Ngo has a bunch here, and Daniel Kokotajlo has his near-prophetic scenario here. I’ve previously tried to do it here (going out with a whimper leading to Bostrom’s “disneyland without children” is one of the most poetic disasters imaginable—great setting for a story), and have a bunch more ideas I hope to get to.
But overall: the LessWrong bubble has a high emphasis on radical AI futures, and an enormous amount of fiction in its canon (HPMOR, Unsong, Planecrash). I keep being surprised that so few people combine those things.
I guess the big problem for someone who tries to do it not in small form is that while you write the story it is already getting old. There are writers who can write a novel in a season, but not many. At least if we talk about good writers. Hm-m-m, did rationalists try to hire Stephen King? :)
Do the stories get old? If it’s trying to be about near-future AI, maybe the state-of-the-art will just obsolete it. But that won’t make it bad necessarily, and there are many other settings than 2026. If it’s about radical futures with Dyson spheres or whatever, that seems like at least a 2030s thing, and you can easily write a novel before then.
Also, I think it is actually possible to write pretty fast. 2k/day is doable, which gets you a good length novel in 50 days; even x3 for ideation beforehand and revising after the first draft only gets you to 150 days. You’d have to be good at fiction beforehand, and have existing concepts to draw on in your head though
Love pieces that manage to be both funny and thought-provoking. And +1 for fitting a solar storm in there. There is now better evidence of very large historical solar storms than there had been during David Roodman’s Open Phil review in late 2014, have been meaning to write something up about that but other things have taken priority.
I have a writeup on solar storm risk here that could be of interest
In many respects, I expect this to be closer to what actually happens than “everyone falls over dead in the same second” or “we definitively solve value alignment”. Multipolar worlds, AI that generally follows the law (when operators want it to, and modulo an increasing number of loopholes) but cannot fully be trusted, and generally muddling through are the default future. I’m hoping we don’t get instrumental survival drives though.
The “Warring nanobots in the upper atmosphere” thing doesn’t actually make sense.
The zaps of light are diffraction limited. And targeting at that distance is hard. Partly because it’s hard to tell between an actual animal and a bunch of nanobots pretending to be an animal. So you can’t zap the nanobots on the ground without making the ground uninhabitable for humans.
The “California red tape” thing implies some alignment strategy that stuck the AI to obey the law, and didn’t go too insanely wrong despite a superintelligence looking for loopholes (Eg the social persuasion infrastructure is already there. Convince humans that dyson sphere are pretty and don’t block the view?).
There is also no clear explanation of why someone somewhere doesn’t make a non-red-taped AI.
You have restored my faith in LessWrong! I was getting worried that despite 200+ karma and 20+ comments, no one had actually nitpicked the descriptions of what actually happens.
In practice, if you want the atmospheric nanobots to zap stuff, you’ll need to do some complicated mirroring because you need to divert sunlight. And it’s not one contiguous mirror but lots of small ones. But I think we can still model this as basic diffraction with some circular mirror / lens.
Intensity I=ceEπr2, where E is the total power of sunlight falling on the mirror disk, r is the radius of the Airy disk, and ce is an efficiency constant I’ve thrown in (because of things like atmospheric absorption (Claude says, somewhat surprisingly, this shouldn’t be ridiculuously large), and not all the energy in the diffraction pattern being in the Airy disk (about 84% is, says Claude), etc.)
Now, E=π(D2)2L, where D is the diameter of the mirror configuration, L is the solar irradiance. And r=θl, where l is the focal length (distance from mirror to target), and θ≈1.22λ/D the angular size of the central spot.
So we have I≈ceLD41.222×4λ2l2, so the required mirror configuration radius D=4√1.222×4Iλ2l2ceL.
Plugging in some reasonable values like λ≈5×10−7 m (average incoming sunlight—yes the concentration suffers a bit because it’s not all this wavelength), I=107 W/m^2 (the level of an industrial laser that can cut metal), l=104 m (lower stratosphere), L=1361 W/m^2 (solar irradiance), and a conservative guess that 99% of power is wasted so ce=0.01, we get D≈18m (and the resulting beam is about 3mm wide).
So a few dozen metres of upper atmosphere nanobots should actually give you a pretty ridiculous concentration of power!
(I did not know this when I wrote the story; I am quite surprised the required radius is this ridiculously tiny. But I had heard of the concept of a “weather machine” like this from the book Where is my flying car?, which I’ve reviewed here, which suggests that this is possible.)
I don’t really buy this, why is it obvious the nanobots could pretend to be an animal so well that it’s indistinguishable? Or why would targeted zaps have bad side-effects?
Yeah, successful alignment to legal compliance was established without any real justification halfway through. (How to do this is currently an open technical problem, which, alas, I did not manage to solve for my satirical short story.)
This is a good point, especially since high levels of emotional manipulation was an established in-universe AI capability. (The issue described with the Dyson sphere was less that it itself would block the view, and more that building it would require dismantling the planets in a way that ruins the view—though now I’m realising that “if the sun on Earth is blocked, all Earthly views are gone” is a simpler reason and removes the need for building anything on the other planets at all.)
Yep, this is a plot hole.
These nanobots are in the upper atmosphere, possibly with clouds in the way, and the nanobot fake humans could be any human to nanobot ratio. Nanobot internals except human skin and muscles. Or just a human with a few nanobots in their blood.
Because nanobots can be like a bacteria if they want. Tiny and everywhere. The nanobots can be hiding under leaves, cloths, skin, roofs etc. And even if they weren’t, a single nanobot is a tiny target. Most of the energy of the zap can’t hit a single nanobot. Any zap of light that can stop nanobots in your house needs to be powerful enough to burn a hole in your roof.
And even if the zap isn’t huge, it’s not 1 or 2 zapps, it’s loads of zapps constantly.
I think this is a story about how AI was severely regulated by the courts, leading to incentives away from AI progress combined with global catastrophic events setting humanity’s technology back by a lot, so AIs didn’t kill us.
It’s nice to read some realistic science fiction.
Too much runs into the very real issue that truth is stranger. 😉
This was wonderful. Anyone have recommendations for similar writing?
Here’s 4:
https://www.lesswrong.com/posts/kj4jW9DxtKQBJbapn/stanislav-petrov-quarterly-performance-review
https://www.astralcodexten.com/p/every-bay-area-house-party?utm_campaign=posts-open-in-app&triedRedirect=true
https://www.lesswrong.com/posts/pL4WhsoPJwauRYkeK/moses-and-the-class-struggle
https://www.lesswrong.com/posts/aRBAhBsc6vZs3WviL/ommc-announces-rip
Good list!
I personally really like Scott Alexander’s Presidential Platform, it hits the hilarious-but-also-almost-works spot so perfectly. He also has many Bay Area house party stories in addition to the one you link (you can find a bunch (all?) linked at the top of this post). He also has this one from a long time ago, which has one of the best punchlines I’ve read.
I agree about the punchline. Chef’s kiss post
https://www.lesswrong.com/posts/pk9mofif2jWbc6Tv3/fiction-a-disneyland-without-children
Thanks for advertising my work, but alas, I think that’s much more depressing than this one.
Could make for a good Barbie <> Oppenheimer combo though?
Also this very recent one: https://www.lesswrong.com/posts/6h9p6NZ5RRFvAqWq5/the-summoned-heroine-s-prediction-markets-keep-providing
“Twitter, I mean X” in 2045 had me dying. So did the California red tape part. Awesome job.
pretty sassy abbreviations spiced in there.’Đ
I’ve expected the hint of
> My name is Anthony. What would you like to ask?
to show it Anthony was an LLM-based android, but who knows.?.