You’re doing the Lord’s work.
I think you’d have affinity with https://aisafetyawarenessproject.org/
MAN this sounds fun. As a kid hoping to be a writer, I dreamt of biking across America and talking to whomever.
I’m sure pdku is helping with this, but it could be high-impact for you to get clickbaitier and more popular. If money is a pullable lever for this stuff, you should apply for some sort of micro-grant. (Or DM me.)
I.M.J. McInnis
Karma: 540
Not loving that the doom AI in IABIED was named Sable, and Anthropic just released Fable. Like, what else was being considered? The Germinator? PAL-9000?
Bleak if true!
I would be interested in hearing from Anthropic employees here, but I imagine it’s the usual reason: they have deemed it inexpedient to do so, either because they believe that they need to “sound normal” or because they in fact don’t want the type of regulation that awareness of x-risk would produce.
It was spooky how much of my own commercial thinking I recognized here. (We just launched! https://www.withai.co/). Am meant to be sending emails right now, so I’ll resist the urge to write my little manifesto, but I’m excited for this era of work (despite being gloomy about superintelligence)
hm. it worked for the early Christians...
in one case, Claude managed to infer a user’s inner struggles by observing the user’s multiple queries relating to catholic relics and saints, and other groups of queries relating to gay male models
Claude is leaning on stereotypes here. Without queries about the sinfulness-or-not of homosexuality, this may well just be a pious, gay Catholic. I am friends with many!
This currently links to “https://armiercansformoskovitz.com/″, which does not resolve
These all look rather… linear. Worrisome! Though it would be better if we could see 0 and see more backhistory.
OpenAI models (particularly 5-thinking) have long had a fetish for jargon, cramming their sentences as full of it as the situation allows (and sometimes fuller).
A fun riddle I was shocked to see the gippities solve without extended thinking or much yapping, even. I gave up on the third!
Here’s the riddle, as stated to Opus 4.6 on an empty context window. “Consider single-word-name countries. An inclusive pair is when the name of one country is contained in another wholly. There are three such pairs. Find them.”
I was surprised not only by how quickly it solved it, but also by the lack of thinking tokens. Gemini 3.1 Fast also did it. And by the unusual order in which the solutions were produced, in the exact reverse order most people find them.
(Answer below)
.
.
.
.
.
.
.
.
Despite the anemia of this law, it has still enraged the super PACs like nothing else. Imagine how they’ll react when we pass a real law.
This is (instinctively) quite frightening, particularly the “no we NEED to spy on American citizens at scale, and we NEED killer robots.” On the other hand… I guess it shows that the government isn’t always and everywhere in the pocket of AI developers? Or maybe it just further shows that the government is in the pocket of certain other AI developers...
If there were strong and broad political will for treating AI like nuclear power and slowing it down arbitrarily much to keep risks low, the situation might be different. But that isn’t the world we’re in now, and I fear that “overreaching” can be costly.
I think it would make a nontrivial contribution to that ‘strong and broad political will’ if Dario were to come out and say “actually, sorry about all that deliberate Overton-window-closing I did in previous writings. In fact, political will is not a totally exogenous oh-well thing, but it is the responsibility of frontier developers to inculcate that political will by telling the public that a pause is possible and desirable, instead of a dumb lame thing not even worth considering. So now we’re saying loud and clear: a pause is possible and desirable, and the world should work toward it as a Plan A!”
I’m being deliberately cartoonish here, but you get the point. If incentives are forcing Anthropic to abandon things that are good for human survival––which occurrence was, no offense, completely obvious from day one––Anthropic should be screaming from the rooftops, Help!! Incentives are forcing us to abandon things that are good for human survival!!
If this is a crux for you––if you/Anthropic think a pause is so undesirable/unlikely that it’s important for the safety of the human race to publicly disparage the possibility of a pause (as Dario opens many of his essays by doing)––please say so! Otherwise, this lily-livered, disingenuous, “oh no, the incentives! it’s a shame incentives can never be changed!” moping will give us all an undignified death.
To be clear, I’m not actually mad about the weakening of the RSP; that was priced in. I suppose I’m glad it’s stated, in case there were still naïfs who thought A Good Guy With An AI could save us. It’s far more virtuous than outright lying, as every other company (to my knowledge) does (more of).
Also, although you seemed to try to answer “What is the point of making commitments if you can revise them any time?” You really just replied “Well, actually these commitments were inconvenient to revise, and in fact they should be more convenient to revise, albeit not arbitrary convenience.” Forgive me if I am not reassured!
I respect your work a lot, Holden. You’ve done great things for humanity. Please don’t lose the forest for the trees.
My main concern is that, in a targeted persuasion-of-the-powerful campaign, you want folks who are very good at persuading the powerful. By talent, this seems largely to draw from the “intellectually honest fringe” (where I and my friends live), who (as I understand it) haven’t been successful in “targeted persuasion of fancy, respectable people.” I really like the idea, though! I think this might be one of the most under-resourced causes in AI safety, and I plan to donate to help with the bridge funding. But—what should reassure me about my worries, that y’all are odd and not-well-connected, therefore being ill-suited to persuade all these important people?
Please flag up front how and how much you used LLMs for the writing. When the prose and structure is this LLMy, I’m forced to assume the ideas are LLMy too, so I don’t read it.
If anyone has influence at a venture capital firm, it would be really really worthwhile to make this one of their big “calls for startups.”
On the one hand, I am glad to see such awareness and honesty about the risks. On the other hand, I remain furious at Dario’s ceaseless insistence that a pause or slowdown is completely impossible (and so, it is implied) not even worth trying for.
It’s stated at the bottom of the webpage. After a few paragraphs at the start I was like… “ok surely this is AI generated” and popped it in a few detectors. i happened to scroll to the bottom to see how long the dang thing was and saw the disclaimer. i wish this former lab insider—who surely has money and at least one eloquent friend—could’ve paid another human being to rewrite it. but alas!
Nope. The Cycle Double Cover Conjecture was resolved true. https://cdn.openai.com/pdf/04d1d1e4-bc75-476a-97cf-49055cd98d31/cdc_proof.pdf