This currently links to “https://armiercansformoskovitz.com/″, which does not resolve
I.M.J. McInnis
Karma: 499
These all look rather… linear. Worrisome! Though it would be better if we could see 0 and see more backhistory.
OpenAI models (particularly 5-thinking) have long had a fetish for jargon, cramming their sentences as full of it as the situation allows (and sometimes fuller).
A fun riddle I was shocked to see the gippities solve without extended thinking or much yapping, even. I gave up on the third!
Here’s the riddle, as stated to Opus 4.6 on an empty context window. “Consider single-word-name countries. An inclusive pair is when the name of one country is contained in another wholly. There are three such pairs. Find them.”
I was surprised not only by how quickly it solved it, but also by the lack of thinking tokens. Gemini 3.1 Fast also did it. And by the unusual order in which the solutions were produced, in the exact reverse order most people find them.
(Answer below)
.
.
.
.
.
.
.
.
Despite the anemia of this law, it has still enraged the super PACs like nothing else. Imagine how they’ll react when we pass a real law.
This is (instinctively) quite frightening, particularly the “no we NEED to spy on American citizens at scale, and we NEED killer robots.” On the other hand… I guess it shows that the government isn’t always and everywhere in the pocket of AI developers? Or maybe it just further shows that the government is in the pocket of certain other AI developers...
If there were strong and broad political will for treating AI like nuclear power and slowing it down arbitrarily much to keep risks low, the situation might be different. But that isn’t the world we’re in now, and I fear that “overreaching” can be costly.
I think it would make a nontrivial contribution to that ‘strong and broad political will’ if Dario were to come out and say “actually, sorry about all that deliberate Overton-window-closing I did in previous writings. In fact, political will is not a totally exogenous oh-well thing, but it is the responsibility of frontier developers to inculcate that political will by telling the public that a pause is possible and desirable, instead of a dumb lame thing not even worth considering. So now we’re saying loud and clear: a pause is possible and desirable, and the world should work toward it as a Plan A!”
I’m being deliberately cartoonish here, but you get the point. If incentives are forcing Anthropic to abandon things that are good for human survival––which occurrence was, no offense, completely obvious from day one––Anthropic should be screaming from the rooftops, Help!! Incentives are forcing us to abandon things that are good for human survival!!
If this is a crux for you––if you/Anthropic think a pause is so undesirable/unlikely that it’s important for the safety of the human race to publicly disparage the possibility of a pause (as Dario opens many of his essays by doing)––please say so! Otherwise, this lily-livered, disingenuous, “oh no, the incentives! it’s a shame incentives can never be changed!” moping will give us all an undignified death.
To be clear, I’m not actually mad about the weakening of the RSP; that was priced in. I suppose I’m glad it’s stated, in case there were still naïfs who thought A Good Guy With An AI could save us. It’s far more virtuous than outright lying, as every other company (to my knowledge) does (more of).
Also, although you seemed to try to answer “What is the point of making commitments if you can revise them any time?” You really just replied “Well, actually these commitments were inconvenient to revise, and in fact they should be more convenient to revise, albeit not arbitrary convenience.” Forgive me if I am not reassured!
I respect your work a lot, Holden. You’ve done great things for humanity. Please don’t lose the forest for the trees.
My main concern is that, in a targeted persuasion-of-the-powerful campaign, you want folks who are very good at persuading the powerful. By talent, this seems largely to draw from the “intellectually honest fringe” (where I and my friends live), who (as I understand it) haven’t been successful in “targeted persuasion of fancy, respectable people.” I really like the idea, though! I think this might be one of the most under-resourced causes in AI safety, and I plan to donate to help with the bridge funding. But—what should reassure me about my worries, that y’all are odd and not-well-connected, therefore being ill-suited to persuade all these important people?
Please flag up front how and how much you used LLMs for the writing. When the prose and structure is this LLMy, I’m forced to assume the ideas are LLMy too, so I don’t read it.
If anyone has influence at a venture capital firm, it would be really really worthwhile to make this one of their big “calls for startups.”
On the one hand, I am glad to see such awareness and honesty about the risks. On the other hand, I remain furious at Dario’s ceaseless insistence that a pause or slowdown is completely impossible (and so, it is implied) not even worth trying for.
It’s stated at the bottom of the webpage. After a few paragraphs at the start I was like… “ok surely this is AI generated” and popped it in a few detectors. i happened to scroll to the bottom to see how long the dang thing was and saw the disclaimer. i wish this former lab insider—who surely has money and at least one eloquent friend—could’ve paid another human being to rewrite it. but alas!
I am surprised you didn’t mention the fact that the whole thing was paraphrased to preserve anonymity by Opus 4.5. (Which really stood out to me! When I first read it, I assumed it was AI-generated, and I was disconcerted to see such quality of thought coming with such a slopreek to the prose.)
MalcolmMcLeod’s Shortform
AI company heads should sign the Superintelligence Statement (by their professed values)
Elon Musk, Demis Hassabis, and Dario Amodei have now all said that they would slow down if they could (or in Dario’s case, “wish we had more time”). They should sign the Statement on Superintelligence—there is not much daylight between their casual remarks in public and what that statement proposes. Employees should pressure their leadership to sign this; we need to make common knowledge so that it’s not so easy for them to say “but oh that will never happen.”
I don’t know how sincere they are about this desire. They sure don’t talk about it much, and when they do, they are so insistent that coordination could never possibly happen, which is an awfully convenient belief. “It is difficult to get a man to understand something when his [billion$ and global status] depends upon his not understanding it.”
Sam Altman and Mark Zuckerberg should sign it too, of course. As should everyone! But it seems much easier to get Musk, Hassabis, and Amodei to do so.
If you’re reading this and can exert any pressure on them to do so (or generally promoting signing it within your company), please please do so.
I expect some measure of “speciation” to occur, but not as a result of money-lack. Rather, some cultures will reject these techs, and as the cultures that accept these techs drift biologically farther from “base humans,” the more entrenched the social and cultural divisions will be.
this is the most lesswrong thing i’ve ever seen. never change
100%! I encourage you to do so. have a soda to give yourself something to do with your hands. give yourself permission to *act drunk*, i.e., be loud, direct, shameless, goofy, emotional, etc.----other people’s drunkenness can easily “rub off” on you.
love how game you are. and admittedly it differs regionally, but this can / should / does work. you haven’t seen people mingle with others because (i’m guessing) it’s hard to tell when “two groups are mingling” in a way that’s distinct from a single group hanging out. (also in fact people do this MUCH less than they used to, but they’re not much less open to it.)
i recommend college or post-college bars, followed by mixed-gender dive bars. or basement shows / unofficial bars. cocktail places are probably bad for this, they are real into Adult Alienation TM. fanciness is generally a barrier. the minglability can be roughly sussed out from the alcohol:
punch (caution!!) > seltzers and cheap beer > craft beers, whiskeys, and applebees-tier cocktails > cocktails > wine
but ensure that the bar has quiet and also has loud. that’s the most important part. the presence of darts, pool, or food also helps give “handles” for conversation.
as for whom to bring and who is game for this… hard to say. varies by friends. the cost of asking is zero, though. (if you have friends who make the cost of asking nonzero, you have exhausting friends!) but yeah, if you have rat-y, great books-y friends, tell them “i am explicitly gonna go try to talk to folks at bars, as i think i’m missing something about the world. we’ll get smashing drunk. it’ll be great.” and if they are not into that… well, try alone, i guess? also get some more adventuresome friends!
similarly, the cost of asking people who Aren’t About It is zero. such folks are, in bars and high drunkenness, really rare! most folks are friendly! (in my experience, which is Virginia, NJ and NYC.) Do not underestimate the lubricant effect of alcohol. but also, if you don’t like the other group, it is SO EASY to take advantage of noise, getting something from the bar, etc., to detach.
if you’re just making the rounds, speaking to one group after another… this is unlikely, but also, NOBODY IS WATCHING. it is so hard to keep tabs on strangers in a bar, even if you’re trying!
and how do you talk and get to know people when the music is so loud—go somewhere loud-ish—such that folks are forced to speak up and perhaps articulate more—but not somewhere that’s so loud it’s “clearly for dancing.” (it is possible to have incredibly compressed shouted conversations, but they’re not very rewarding most of the time.)
GENERAL PRINCIPLES: men are easier to approach than women. people afford you more social latitude if you’re drunk and if they’re drunk. to a first approximation, nobody is paying attention to you unless you talk to them. and everyone loves to talk about themselves.
if you live in NYC, New Haven, or Princeton, hit me up and i am happy to take you. EDIT: nvm I see you’re Toronting atm. I cannot speak to Canadians’ dispositions, but surely the politeness helps!
Claude is leaning on stereotypes here. Without queries about the sinfulness-or-not of homosexuality, this may well just be a pious, gay Catholic. I am friends with many!