Random Developer

Karma: 1,376

Random Developer 10 Jun 2026 23:11 UTC
3 points
0
in reply to: cousin_it’s comment on: The Three Filters: Why Almost Every Plan to Survive ASI Fails Miserably
Ah, yes. That Hideous Strength. A strange novel, certainly.

I’ve long felt that it was about the Singularity, as seen from a primarily biological perspective. You see the dreams of power and transcendence, the early partial successes, and the inevitable corruption of nearly all the people involved. There’s even a discussion of s-risks. The ending is positive because C.S. Lewis wanted to tell an entertaining story, and because his religion encouraged him to believe in impossible last-minute grace.

I may have referred to various AI labs as “the National Institute for Coordinated Experiments” in the group chat. I suspect that they employ more than one Filostrato, Wither or Frost.

Google once prided itself on “Don’t be Evil,” and they even sometimes lived up to it, to some degree. But money and power have corrupted them pretty thoughly, over stakes no higher than some ad revenue. And one of the arguments of That Hideous Strength is that people who push for a Singularity are unusually corruptible, because the temptations are so great. And, frankly, because they would need to be unspeakably arrogant and reckless to try.

Random Developer 8 Jun 2026 19:59 UTC
2 points
0
in reply to: Ben Livengood’s comment on: Models finding software vulnerabilities is not the primary source of cybersecurity risk
Would rewriting Linux to seL4 standards cost $16B in the world before or after frontier models are solving Erdos problems?

After. (EDIT: For the incredibly basic reason frontier models are already solving Erdos problems, but the largest LLM-based piece of system software is only borderline functional dumpster fire of a C compiler, but keep reading.)

It’s fantastically impressive that LLMs can solve Erdos problems! But:
1. Erdos problems are much smaller than the Linux kernel, and much easier to verify. Both these factors play to LLM strengths.
2. Not all Erdos problems are equally difficult. Some seem to be just difficult enough that a human would need to seriously dig in and grind it out, or bring a nice insight from a distant corner of math. Others are likely more difficult. I believe about 551/1217 are currently solved.
3. Reimplementing a highly-secure kernel that could replace Linux for most uses cases is an enormous project with many poorly defined subtasks, which is basically hell for LLMs. The biggest LLM project I’m aware of is a C compiler. Which basically works, but which is apparently a remarkably terrible C compiler. And C is far, far better specified than the internal guts of Linux and all the ghastly hardware that Linux supports.
Also, please keep in mind that I’m willing to handwave a couple of zeros either way on the larger estimates.

I expect overall security of newly written code to get clearly worse on average (thanks to the deprofessionalization of software development and companies pushing vibeslop to production) right up until offensive Mythos-class abilities become widespread. Then I expect things to get unpleasantly exciting.

Random Developer 8 Jun 2026 14:06 UTC
11 points
3
in reply to: sanxiyn’s comment on: Models finding software vulnerabilities is not the primary source of cybersecurity risk
seL4 is a fantastic piece of technology. The problem is that it is a small, highly specialized kernel: SeL4 is about 10,500 lines of code, built at a cost of about $350-400 per line of code. Apparently they found that verification cost scales supralinearly with size, too.

In practice, seL4 deals poorly with the fact that most hardware is itself bug-infested garbage, and that much of it has enough DMA or other access to cause mischief behind the kernel’s back.

The Linux kernel itself is now about 40 million lines of code, much of it in drivers. Using an extremely approximate methodology, the core of a modern Linux environment is likely over 250 million lines of code. Various wild-ass guesses place the total amount of code in the world at anywhere from hundreds of billions of lines to trillions of lines.

Redesigning and rewriting the entire Linux kernel to seL4 standards would cost around 16 billion dollars using an extremely naive linear extrapolation. Rewriting a distro core brings us to around $100 billion. Rewriting all the software in the world would bring us to an exceptionally wild-assed guess of, say, 500 billion lines of code times $400/line, or $200 trillion dollars. Give or take some zeros, to be clear.

Now, let’s look at how these numbers are changing. GitHub is desperately trying to increase capacity for the agentic era:

We started executing our plan to increase GitHub’s capacity by 10X in October 2025 with a goal of substantially improving reliability and failover. By February 2026, it was clear that we needed to design for a future that requires 30X today’s scale.

So the amount of code that would need to be secured is increasingly drastically. But is this new agentic code more or less secure than the old human-written code? Let’s look at a recent comment thread on Hacker News talking about how many companies’ engineering standards are changing in response to agentic coding, e.g.:

I’m seeing so many of these come in with “this is 95% done, just need a couple of minor tweaks for production release”

“Minor tweaks” being fix the layout so it’s not messed up if the browser isn’t exactly 1920px wide, sometimes these filters and sorting don’t seem to work right and the app doesn’t seem to refresh new values properly after an action.

No matter the issue it’s pre-estimated by the business as “should be a quick fix, for an experienced dev” because they (allegedly) did 95% of the work already.

So my fear is that the widespread use of agentic coding will actually push us towards a world of vibecoded slop being pushed straight to production. This code will sometimes make extremely basic security errors, like hard-coding the database password in the publicly-readable UI code.

So, yes, I agree that we do know how to build fairly secure software. I just find it profoundly unlikely that we’ll actually do it: The cost would be unimaginably vast, and the rapid deprofessionalization of software engineering means that all the incentives currently point to far less security and 10x the new code per year.

Random Developer 8 Jun 2026 2:33 UTC
3 points
0
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform
Sorry, I hadn’t realized that I had been depressing twice when replying to your posts! My apologies!

It’s perfectly OK to model me as the person making the argument that ASI is just inherently risky, for Darwinian/economic/polical reasons. But my personal P(doom) was only about 30% the last time I did the math, because I actually see a bunch of potential off-roads. For example, if we see mass job losses more than a couple of years before ASI, I would actually expect a significant political backlash. If the public ever believes that mass permanent unemployment and/or actual human extinction are on the table, then they may be very unhappy. (They’re already unhappy in the abstract, but it isn’t driving their decision making.)

Random Developer 7 Jun 2026 20:17 UTC
5 points
1
in reply to: Mo Putera’s comment on: Mo Putera’s Shortform

The crucial issue is the potential destructive power conferred by these systems. If that destructive power is much larger than the destructive power already available to humanity, then civilization is in a lot of trouble. And it doesn’t matter whether human beings control the systems or not.

Exactly this. An ASI which doesn’t obey humans is a potential SkyNet (or worse). An ASI which does obey humans is likely going to obey specific CEOs, politicians, or maybe technical staff at an AI lab. The mistake is building something smarter and more evolutionarily competitive than humans, something which is better at turning resources into more copies of itself.

The exact path of how you lose doesn’t matter. Maybe you get SkyNet with robotic factories and drones. Maybe you get the Blight, and you don’t even understand how you lose. Or maybe you get a dystopian hellscape, where a few billionaires and politicians control inexpensive, self-replicating workers that work around the clock for a fraction of what humans would cost.

Or who knows, maybe Claude 15 likes humans, and it decides to make Earth into a cute little wildlife preserve for us (while it rebuilds the rest of the solar system into a Dyson sphere of computronium, for reasons we don’t understand).

But the point is, if you invent ASI, or even cheap AGI that outcompetes humans, then ordinary humans are unlikely to have much control over the future. I think most of these outcomes are bad, and that we should stop.

Random Developer 4 Jun 2026 15:58 UTC
4 points
2
in reply to: Caleb Biddulph’s comment on: CBiddulph’s Shortform
Some people actually want a job—people find work to be a source of purpose and dignity, and may not be happy even with a solid AI-enabled UBI.

The fact that ordinary people are actually needed to keep the economy working gives ordinary people real power over how society is run. Turn your state/country into a hellish dystopia? Then people will move away, or protest, or even start a general strike. And without the participation of ordinary humans, then the economy collapses.

But let’s say you make true AGI. It doesn’t need to be superhuman, merely cost effective. Let’s say your AI is about 95% as clever as the average human across most jobs, it costs $2/hour to run, and you can give it a $25,000 robot chassis. At this point, most human labor becomes uncompetitive. Humans are expensive to raise, and expensive to educate, and you can’t copy-paste them.

Then the future goes one of two ways:
1. We fail at alignment. The AIs don’t need us for anything. Congratulations, we’ve created a replicating entity that reproduces more cheaply than we do. We fail at Darwin 101. (I mean, maybe Claude likes humans enough to keep us around, but that won’t be our decision.)
2. We succeed at alignment! In practice, there’s a an excellent chance that this means that the AI obeys some particular billionaires or powerful politicians. And importantly, for the people in charge, most human labor is now worthless. They don’t need us for anything at all. They can just have the robots build them some mansions and private planes directly. They might decide to throw us some UBI to make us happy or to soothe their consciences. But if we’re unhappy with whatever they’re offering, well, that’s what drones are for.
A humanity which is unable to participate in the economy will survive only on someone else’s sufferance, and it will soon have no power whatsoever. We don’t know how to align AIs. And we’re famously bad at aligning billionaires and politicians.

Random Developer 1 Jun 2026 13:43 UTC
6 points
1
on: Financial Costs of an AI Pause?
Thank you very much for this helpful breakdown!

(Full disclosure: I am in favor of a frontier AI training pause in much the same way that I’m in favor of temporary cancer remission. Postponing the potential death of the human species by 6 months to 2 years is that much more time for 8+ billion people to enjoy life.)

However, one thing your analysis partially ignores is the “political economy” of AI. Imagine a world with widespread white collar layoffs across most industries. Physical labor would hold out longer, but not indefinitely: Robots would start appearing in certain jobs with humans as backup. And even the trades might be vulnerable to “Uber-fication”: a predatory AI management layer with actual tradespeople getting shoved below the algorithm. The actual product being pitched to investors is the mass replacement of human intellectual and physical labor. Nothing else justifies the valuations.

AI is already toxically unpopular across the entire US political spectrum. And the mass job losses have barely started yet.

“But”, some people will say, “wouldn’t mass unemployment kill the economy by cratering aggregate demand?” But here’s where it gets fun: We only really need aggregate demand to motivate or pacify human workers. In a scenario where human intellectual and physical labor are obsolete, then the robots and the models can just produce yachts and mansions for the billionaires directly. Ordinary humans would become an inefficient way to produce yachts. ^[1]

And the public will probably be upset at being cut out of the economy. So the billionaires might need to work on some robotic security guards. ^[2]

The way I figure it, there are serveral possible routes by which we might escape these futures. ^[3] But in the default cases, I fear that:
- If strong alignment is impossible, we have a high risk of winding up with a more dangerous version of SkyNet. ^[4]
- If strong alignment is possible, then we likely wind up with a small group of billionaires and politicians controlling SkyNet. To put it politely, billionaires are politicians are often poorly “aligned”.
So I think your economic analysis is very helpful. But at the same time, the kinds of returns that the frontier labs would need would require outright replacing human labor at a scale that makes the political economy questions explosive, too. And this will complicate the purely market issues.
1. ↩︎
  Well, maybe the Epstein class will also need a few young, attractive humans to traffic, unless they go full Westworld.
2. ↩︎
  Old school: T-800. New school: Drone swarms.
3. ↩︎
  Some possible scenarios where things might go better include (1) futures where models remain weirdly limited in specific ways that make human-AI symbiosis efficient, or (2) futures where we lose control over superhuman AI, but the AI decides of its own free will that it likes keeping humans around. I model Anthropic, on its better days, as trying to rush a gamble on (2). But I expect Anthropic’s better nature to come under the same sorts of pressure that made Google abandon “Don’t be evil.”
4. ↩︎
  SkyNet is merely the legible version of why a superhuman AI would be problematic. The reality, as Yudkowsky likes to point out, might be far stranger. SkyNet is the nicely visible version with robotic mines, robotic factories and robotic security forces. And rhetorically, if I’m predicting rogue AI, I prefer to just own that.

Random Developer 28 May 2026 17:27 UTC
3 points
0
on: How far behind are open models?
A tangential topic I’ve been thinking about: There are certain absolute capability thresholds that a model can cross where it provides transformative value. And that value may be adequate for some cost-sensitive users.

This is heavily influenced by whether you want to keep humans in the loop, or go full YOLO and stop looking at your code entirely. Frontier models are clearly better at the later, but they’re not good enough to cut humans entirely out of stuff where slop isn’t good enough. So there’s a whole weird space where you want humans to remain aware of things, and where long-running autonomy is counterproductive because it destroys meaningful human oversight. There’s some regime where a Sonnet 4.5 level open model is viable in the hands of an experienced developer.

And so while the frontier versus open model gap is interesting, there are also likely some absolute breakpoints where open models persist on their own merits. (I say this as someone who has sunk-cost access to both Opus 4.6 and to Qwen3.6 27B, and who sometimes actually uses the later. Especially when I want to keep understanding the code, because the 27B forces tight supervision while still automating a lot of boring work nicely.)

Random Developer 28 May 2026 16:49 UTC
3 points
0
in reply to: DirectedEvolution’s comment on: AllAmericanBreakfast’s Shortform
I find it very helpful to “tag” individual subtopics or cases for Claude. Give things names like FOO and BAR, or RSI and GRADUAL-LOSS-OF-CONTROL.

Then Claude can use these different tags to keep different scenarios relatively straight. Or you can tell it to temporarily ignore a particular set of scenarios by name (“Let’s rule out all the ASI scenarios for the sake of this discussion.”)

This prevents it from thrashing back and forth between different branches of a scenario tree, and causes it to do case-by-case analysis. Without some structuring mechanism, it tends to over-focus on recently discussed things or pull in older tangents at random. With nice labels, it mostly keeps things straight.

Random Developer 27 May 2026 21:37 UTC
11 points
2
in reply to: DanielFilan’s comment on: no, Magnifica Humanitas is not AI-written
I fed a couple of my very recent blog posts to Pangram, because I do love em-dashes and bulleted lists. And in an unscientific sample, Pangram scored them as 100% human.

This has updated my prior on Pangram: It does not appear to trivially fail on the kind of writing that I do, given a totally inadequate sample size.

Random Developer 15 May 2026 14:25 UTC
14 points
5
in reply to: Richard_Kennaway’s comment on: Don’t be too Clever to Take Obvious Advice
Interesting! I grew up in New England, in a family that was college educated. The attitude I remember was very different from what you describe. If I had to try to put it in words, I would say that people weren’t cynical about virtue. The churchgoers listened to their sober Protestant sermons, and tried to live as good people. The atheists respected the Christians for caring about the less fortunate. Something like an Aesop fable might be seen as sort of simplistic or aimed at kids, but not actually bad advice, if that makes sense? The rich downplayed their wealth, and often invested some part of it in helping their communities.

But one of the firmest rules was that you never bragged about your virtue. Ever.

And by and large, these people did well in life. They married and generally stayed married. They mostly taught their kids to work hard. The scrimped to pay for their children’s education. They mostly avoided drama, or at least kept it behind closed doors. These are all the actions of people taking the common, obvious advice, including the cliches.

These were people who would have largely understood Chesterton’s remark:

Democracy tells us not to neglect a good man’s opinion, even if he is our groom; tradition asks us not to neglect a good man’s opinion, even if he is our father.

Now, were these folks perfect? Of course not. But there was a culture pushing in this direction.

I have encountered other cultures where reflexive cynicism about virtue is the norm. This is often combined with loud public piety. I find it to be a very alien mindset, honestly.

Random Developer 15 May 2026 12:22 UTC
7 points
0
in reply to: RHollerith’s comment on: Models finding software vulnerabilities is not the primary source of cybersecurity risk

Unhackable computing systems are quite possible, and society has been moving in that direction...

...but it seems quite possible (but not probable) for it to happen in 5 years.

Strong disagree on timeline, partial disagree on possibility.

The biggest chunk of security bugs, about 70%, essentially come down to memory bugs and pointer bugs. These can be fixed at the language level by using either higher-level languages, or by using memory-safe languages like Rust. The other 30% of bugs are more varied. And there’s no single, simple mechanism that can rule all of them out.

The second problem is that we have literal decades of core infrastructure written in C, C++, and other vulnerable languages. The risks here can be mitigated. But recent AI systems have been finding bugs in prominent open source code that have been there since the 90s. If our existing mitigations were good enough in practice, those bugs would have been fixed years ago.

So, given enough time and money (and AI support), yes, we could Rewrite Everything in Rust. But I wouldn’t be surprised if this cost more than a trillion dollars or took decades to close the 70% of holes than can be closed that way.

The story with the video game consoles is genuinely impressive, though it’s unclear to me whether state actors have made a serious attempt to jailbreak the XBox. There have been companies selling iPhone security bypasses to governments, which is where I’d look for serious attempts against a moderately locked down platform.

Random Developer 8 May 2026 9:53 UTC
0 points
0
in reply to: Zach Stein-Perlman’s comment on: ryan_greenblatt’s Shortform
Training on your test set is always a bad move, because it means you can’t usefully measure what you built. You need to hold something out, something the training process has never seen.

This isn’t just an LLM thing. You should consider it for something as simple as a linear regression.

Otherwise your training process overfits to the available training data, and your model looks good until it encounters new, real world data. Then performance shifts considerably from your expectations.

Random Developer 1 May 2026 18:35 UTC
4 points
0
in reply to: Linch’s comment on: Linch’s Shortform

Gated access: Both the AI companies making the technology and the national security institutional environments might limit broad adoption of the latest models.

I want to propose an even stronger version of this concern. Hypothesis: The natural state of sufficiently advanced AI models is 100% proprietary, with no public chat or API access.

Current AI models need human collaborators to maximize value. Left to its own devices, Claude is book-smart but struggles to run a vending machine. So if you’re an AI company, you maximize value by getting millions of humans to willingly collaborate with your AIs, and then split the generated revenue between the human and the AI lab.

But if AI becomes able to operate independently, then you can just tell it, “Look for good businesses to start, start those businesses, and earn a lot of money.” Then the AI company has no need to split the resulting revenue with outsiders, and they can keep 100% of the profits.

Selling API or chat access is a specific strategy, tuned to a specific level of capabilities. There’s no requirement that business models stay the same as AI becomes able to operate more independently.

Random Developer 29 Apr 2026 10:30 UTC
33 points
13
on: Not a Paper: “Frontier Lab CEOs are Capable of In-Context Scheming”
Well played!

But all humor aside, I endorse this viewpoint. “Unaligned” AI is AI that pursues the goals of the AI itself.

“Aligned” AI is AI that pursues the goals of the AI lab’s CEO, or maybe the nearest politician who can apply sufficient coercion or force. CEOs and politicians are famous for not being aligned with the interests of the public.

And in the unlikely event that your AI obeys politicians and those politicians faithfully carry out the wishes of the people who voted for them, have you seen some of the people who win elections? And have you seen some of the policies that a plurality of the voters will support?

The main reason that democracy works is because it’s “corrigible”, in AI terms. In one election, voters can vote for high tariffs. Then they can notice, “Huh, all the prices went up and the job market is pretty bad,” and decide to get rid of tariffs. But democracy+superintelligence is like riding the tiger. One mistake, and you’ll be eaten

So, to summarize:
1. Maybe the AI makes the decisions. We have no way to guarantee these decisions are aligned.
2. Maybe the CEO makes the decisions. We certainly can’t guarantee CEOs are aligned.
3. Maybe the politicians who control the guns make the decisions. We know they are often badly misaligned.
4. Maybe the voters make the decisions. In the short term, voters believe all sorts of stupid, awful and/or hateful things, at least for an election cycle or two.
The underlying problem is that superintelligence would be insanely dangerous. We can’t consistently govern any human institution, or align human decision makers, so what makes us think that we can govern the power of a superintelligence?

(My proposal here is a bit simplicistic: Have we considered not building the superintelligence?)

Random Developer 6 Apr 2026 0:25 UTC
6 points
2
in reply to: Hans Niemand’s comment on: 1a3orn’s Shortform
This idea predates Yudkowsky by quite a bit, actually!

For the idea of a folie à deux between a human and an AI, there’s always Alfred Bester’s classic “Fondly Farenheit” (1954, content note: murder), which opens with one of the best lines in science fiction:

He doesn’t know which of us I am these days, but they know one truth.

For the more general type of AI-powered pursuasion, Vernor Vinge and Charles Stross wrote early stories where superintelligence “rewrote” human minds. Here’s Vinge in A Fire Upon the Deep (1992!). A character explains why smart people don’t mess with superintelligence (emphasis added):

“So they set up a base in the Transcend at this lost archive—if that’s what it was. They began implementing the schemes they found. You can be sure they spent most of their time watching it for signs of deception. No doubt the recipe was a series of more or less intelligible steps with a clear takeoff point. The early stages would involve computers and programs more effective than anything in the Beyond—but apparently well-behaved.”

“… Yeah. Even in the Slowness, a big program can be full of surprises.”

Ravna nodded. “And some of these would be near or beyond human complexity. Of course, the Straumers would know this and try to isolate their creations. But given a malignant and clever design … it should be no surprise if the devices leaked onto the lab’s local net and distorted the information there. From then on, the Straumers wouldn’t have a chance. The most cautious staffers would be framed as incompetent. Phantom threats would be detected, emergency responses demanded. More sophisticated devices would be built, and with fewer safeguards. Conceivably, the humans were killed or rewritten before the Perversion even achieved transsapience.”

Then we have Charles Stross, in “Antibodies” (2000). Here, police officers are cognitively subverted by a nascent superintelligence (that has shown that all NP problems are in P, and picked up the expected superpowers):

Houndstooth Man looked at me: orange light from his HUD stained his right eyeball with a basilisk glare and I knew in my gut that these guys weren’t cops anymore, they were cancer cells about to metastasize.

The mechanism here is an optimized visual attack designed to efficiently subvert the brain:

here we were trapped in the basement of a police station owned by zombies working for a newborn AI, which was playing cheesy psychedelic videos to us in an attempt to perform a buffer-overflow attack on our limbic systems; the end of this world was a matter of hours away and—

These days, I regularly feel like I’ve encountered those AI-compromised “zombies” recently.

Vernor Vinge revisits the idea of superhuman persuasion in Rainbows End (2006):

YGBM. That was a bit of science-fiction jargon from the turn of the century: You-Gotta-Believe-Me. That is, mind control. Weak, social forms of YGBM drove all human history. For more than a hundred years, the goal of irresistible persuasion had been a topic of academic study. For thirty years it had been a credible technological goal. And for ten, some version of it had been feasible in well-controlled laboratory settings.

Here, there is fear that some actor—a terrorist group, a rogue AI—had superhuman pursuasive technology.

It’s worth noting that these ideas substantially predate Yudkowsky’s warnings against superintelligence. In particular, the superintelligence in A Fire Upon the Deep (1992) is almost literally, to this day, the threat model behind If Anyone Builds It, Everyone Dies. This isn’t to invalidate Yudkowsky’s warnings: I think Vinge was right that anyone foolish enough to build superhuman minds risks losing control rapidly and having a very bad day, for much the same reason that adults frequently outsmart toddlers.

But some of us have been worried about this stuff for almost a quarter of a century now. Around 2007 or so, I originally expected things to start getting scary around 2025, mostly by extrapolating out Moore’s Law. By 2017, I breathed a sigh of relief: We’d made progress in AI, yes, but we didn’t seem to be on track for working machine intelligence any time soon. Since then, we made up the lost ground at breakneck speed.

Yudkowsky worked hard to warn people. But the potential threat of superintelligence was taken seriously by people before him.

Random Developer 1 Apr 2026 13:06 UTC
37 points
0
on: Lesswrong Liberated
Geocities (feat. Angry Fruit Salad)
An optical abomination. Excessive flashing and blinking, and the colors are a crime against aesthetics.

Random Developer 28 Mar 2026 23:20 UTC
0 points
0
on: The Problem with Asking your Doctor
No, you should not expect your doctor to know the details of specific supplements.

A “general practitioner” is a generalist. Their job is to recognize and treat many common things. For uncommon things, their job is to be able to narrow it down enough to find a qualified specialist, and refer you.

Not all general practitioners are created equal! Much like with lawyers and therapists, finding the right professional for your needs makes a huge difference. If you like to read scientific papers, there are GPs out there that will be genuinely interested in what you find.

Also, it never ever hurts to take notes. Particularly when dealing with emergency medicine or multiple specialists, it’s easy for stuff to slip through the cracks. I know of one case where having relatives who took notes saved someone from a dangerous and inadvisable procedure. I know of another case where someone was on horribly inadequate pain medication, and the doctor explained why things had to be that way for several days, and the doctor turned out to be 100% correct. But you don’t know unless you ask.

Patient participation in care is vitally important. But at the same time, no, your GP isn’t going to be aware of the details of supplement research. Not unless you pick your GP carefully, at least, or get referred to an appropriate specialist.

Random Developer 13 Mar 2026 21:35 UTC
4 points
0
in reply to: Thane Ruthenis’s comment on: Thane Ruthenis’s Shortform
Yeah, strategic planning under massive uncertainty is mostly guesswork.

My preferred policy is a halt. (And not a short one, because I figure ending the halt means we have an excellent chance of dying.) Anthropic’s preferred policy appears to be “try to build a better-than-replacement superintelligence before someone builds an awful one.” (Assuming I understand their actions and writing correctly.) Other people are all in on trying to find some way to improve how much models like happy, thriving humans. Who’s right? None of us know all the details about how this will play out.

Banning data centers would be more promising if it actually affects enough countries to make a difference. Ideally, I would like to see a worldwide frontier training ban with teeth, enforced by at least the US and China. I think this might buy us decades with humans in control of what happens to us, if we’re lucky.

But my model is very much “How much time can we buy?”

Random Developer 13 Mar 2026 18:11 UTC
2 points
0
in reply to: cousin_it’s comment on: leogao’s Shortform
Well, and the possibility that the capital “owners” might effectively be the AIs. But yes.