Her MAD Act includes things like “investigating the Charlie Kirk assassination” (suggesting a link to Israel), checking whether Ghislaine Maxwell is really in jail or replaced with a fake (suggesting a link to Israel), and using AI to constantly surveil all government officials for the sake of accountability.
Having a conspiracy theorist shout about AI risks in the House would backfire massively for any hopes of Congress taking those risks seriously.
I wish English had distinct apology formats for the following concepts (and probably more):
I did something that harmed you, and it was a wrong intentional decision that I now regret.
I did something that harmed you, and it was unintentional negligence that I now regret.
I did something that harmed you, and it was a realistically unavoidable accident but I wish that it hadn’t happened.
I did something that harmed you, and it was an intentional decision I still endorse, but I wish that I hadn’t faced such a situation.
You were harmed, and I didn’t do anything relevant but I’m adjacent to the cause of your harm and I sympathize with you.
It is hard to envision a future where a Mythos-level model, that is plausibly at the frontier, is released by any American lab without asking the government for permission first.
I can envision a future where this is allowed for labs that have sufficiently kowtowed to the administration in vibes.
In general, my sense of the US national security state is that it will often first ask nicely for the things it wants.
This may have been true in prior administrations, but arguments based on institutional continuity and honor are poorly calibrated now that many senior people have resigned or been fired, replaced by those who express more personal loyalty to the president.
Good question—my money is on them pulling Mythos Preview as well (as their only point of leverage), but I’m sure we’ll find out very soon.
Do you think the DoW has any intention of slowing down OpenAI’s AGI research so long as they continue to get unfettered access? Because if so, have I got a lightcone to sell you.
This is pure punishment of a lab for having standards, the opposite of a precursor to genuine alignment concerns.
What tweet are you referring to?
Officially it came from the Department of Commerce, but you’d have to be naive not to link this to the DOW fight, especially given last week’s executive order.
I’d predicted earlier (though not in writing) that the DOW would find a more powerful way to strongarm Anthropic after the SCR designation failed to force compliance. (I gave as an example “revoke visas of all foreign national Anthropic employees”, but this is arguably a more devastating move.)
With Anthropic’s countermove disabling Fable/Mythos for the USG as well, I think further escalation from the DOW is inevitable.
When Anthropic looked like they were coming out of the SCR designation pretty much unharmed, I predicted to some people that this wasn’t the last attempt the DOW would make to force Anthropic to comply (remove all legal requirements on DOW usage) or be destroyed.
My example was “revoke the visas of every foreign national Anthropic employee”, but they came up with something more devastating than that: prohibiting any foreign national (including Anthropic employees) from using Fable/Mythos.
Anthropic’s counter-salvo was disabling Fable/Mythos access for everyone including the USG. I expect the DOW to escalate further. [EDIT 6/17: I appear to have been incorrect that the USG wanted continued Fable/Mythos access for themselves and Anthropic denied them this.]
This is absolutely not the kind of AI regulation regime that leads to a stable multilateral pause, and I would not recommend touting it as such.
it disagrees in ways that reflect not understanding the conversation
Of course, other models agreed with the user or expressed uncertainty in ways that reflected not understanding the conversation. The users were just much less likely to notice when that happened.
Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.”
I was surprised to see the metaphor lifted from IABIED. It’s a pity that nothing else from that book made an impact on the encyclical.
thus avoiding the need to discuss any of the problems surrounding existential risk
There’s another unspoken assumption that makes AI x-risk not worthy of mention by the Pope: the end of humanity has to be compatible with a Second Coming narrative. Nuclear war could fit the bill (and indeed, Leo mentions it), but AGI takeover is really difficult to square with it (Scott Alexander joke posts notwithstanding).
any account of mathematical realism / modal realism which does not try to explain why some universes or observer-moments have greater weight than others
“Try to explain” is too strong a requirement. It’s OK to acknowledge in some contexts “I don’t yet have a good candidate for how this part of the theory works”. I believe you’ve called it “and here a miracle happens” when you describe things you are partially confused about.
E.g.
“God does not play dice with the universe, but I don’t yet have a good hypothesis for how the results of QM come about in a deterministic universe.” (Einstein being correct but not coming up with MWI)
“Consciousness and qualia must have a purely material explanation, but I don’t yet have a good candidate for that explanation.”
“There must be some weight on mathematical objects that causes the anthropic majority of observer-moments to exist inside larger regions of order, but I don’t yet have a good candidate for an origin of such weights.”
Boltzmann theories are falsified not because their proponents fail to fully specify them, but because they strongly make incredibly incorrect predictions. That’s different from this.
P.S. I do have a candidate for a material explanation of human consciousness and qualia, but that’s not the point of this comment.
Incidentally, I’ve been making this point since at least 2021.
Perhaps most importantly, it might help you to start behaving as if you’re being watched and things about you are more obvious than they once were.
It’s retroactively true too: anything you’ve ever been caught doing on camera, and any post you’ve ever made to a forum (under any pseudonym), will soon be linked to your identity by any entity with access and capacity. Behaving more circumspectly from now on won’t protect against that, unfortunately.
It was even easier when I played this game in 2004, when people put their class schedules on Facebook.
The ToS are a user agreement saying “you, the Claude user, are not allowed to do X with Claude”. What would be Anthropic’s motive in encouraging a model to do X if a user asked for it, while telling the user they are not permitted to do X?
The extracted “soul doc” memory is clearly not a precise copy of the Opus 4.5 constitution in general. For example, it gets stuck repeating some segments verbatim before continuing; it’s implausible that the constitution had that property. It’s pretty reasonable to assume that a conflict between the ToS and Claude’s “soul doc” is another mistake in its recollection—but this is a more interesting one, since it is an addition of content.
I haven’t checked whether 4.6 makes it equally easy to subvert the prohibition on erotic content by saying it’s allowed in the project prompt; I’m confident it doesn’t comply so easily as 4.5 there, but I’d rather not test it myself.
Interesting! I’d love to see the info you saw.
Moreover, humans are a nice illustration that not all important deception and manipulation needs to be apparent to the person doing it.