orthonormal

Karma: 18,412

orthonormal 18 Jun 2026 19:37 UTC
28 points
2
in reply to: NunoSempere’s comment on: NunoSempere’s Shortform
Her MAD Act includes things like “investigating the Charlie Kirk assassination” (suggesting a link to Israel), checking whether Ghislaine Maxwell is really in jail or replaced with a fake (suggesting a link to Israel), and using AI to constantly surveil all government officials for the sake of accountability.
Having a conspiracy theorist shout about AI risks in the House would backfire massively for any hopes of Congress taking those risks seriously.

orthonormal 17 Jun 2026 19:06 UTC
17 points
1
on: The Financial Ledger Theory of Apologies
I wish English had distinct apology formats for the following concepts (and probably more):
- I did something that harmed you, and it was a wrong intentional decision that I now regret.
- I did something that harmed you, and it was unintentional negligence that I now regret.
- I did something that harmed you, and it was a realistically unavoidable accident but I wish that it hadn’t happened.
- I did something that harmed you, and it was an intentional decision I still endorse, but I wish that I hadn’t faced such a situation.
- You were harmed, and I didn’t do anything relevant but I’m adjacent to the cause of your harm and I sympathize with you.

orthonormal 17 Jun 2026 17:19 UTC
5 points
5
on: The Once And Future Fable #3: Fix This Code
It is hard to envision a future where a Mythos-level model, that is plausibly at the frontier, is released by any American lab without asking the government for permission first.
I can envision a future where this is allowed for labs that have sufficiently kowtowed to the administration in vibes.

orthonormal 16 Jun 2026 1:51 UTC
5 points
3
in reply to: RedMan’s comment on: The Once And Future Fable #2
In general, my sense of the US national security state is that it will often first ask nicely for the things it wants.
This may have been true in prior administrations, but arguments based on institutional continuity and honor are poorly calibrated now that many senior people have resigned or been fired, replaced by those who express more personal loyalty to the president.

orthonormal 13 Jun 2026 17:15 UTC
2 points
0
in reply to: Petropolitan’s comment on: US government directive to suspend access to Fable 5 and Mythos 5
Good question—my money is on them pulling Mythos Preview as well (as their only point of leverage), but I’m sure we’ll find out very soon.

orthonormal 13 Jun 2026 17:08 UTC
8 points
2
in reply to: Haiku’s comment on: orthonormal’s Shortform
Do you think the DoW has any intention of slowing down OpenAI’s AGI research so long as they continue to get unfettered access? Because if so, have I got a lightcone to sell you.
This is pure punishment of a lab for having standards, the opposite of a precursor to genuine alignment concerns.

orthonormal 13 Jun 2026 17:05 UTC
2 points
0
in reply to: yams’s comment on: orthonormal’s Shortform
What tweet are you referring to?

orthonormal 13 Jun 2026 3:36 UTC
10 points
14
in reply to: Mitchell_Porter’s comment on: US government directive to suspend access to Fable 5 and Mythos 5
Officially it came from the Department of Commerce, but you’d have to be naive not to link this to the DOW fight, especially given last week’s executive order.

orthonormal 13 Jun 2026 2:39 UTC
16 points
4
on: US government directive to suspend access to Fable 5 and Mythos 5
I’d predicted earlier (though not in writing) that the DOW would find a more powerful way to strongarm Anthropic after the SCR designation failed to force compliance. (I gave as an example “revoke visas of all foreign national Anthropic employees”, but this is arguably a more devastating move.)
With Anthropic’s countermove disabling Fable/Mythos for the USG as well, I think further escalation from the DOW is inevitable.

orthonormal 13 Jun 2026 2:35 UTC
67 points
36
on: orthonormal’s Shortform
When Anthropic looked like they were coming out of the SCR designation pretty much unharmed, I predicted to some people that this wasn’t the last attempt the DOW would make to force Anthropic to comply (remove all legal requirements on DOW usage) or be destroyed.
My example was “revoke the visas of every foreign national Anthropic employee”, but they came up with something more devastating than that: prohibiting any foreign national (including Anthropic employees) from using Fable/Mythos.
~~Anthropic’s counter-salvo was~~ ~~disabling Fable/Mythos access for everyone including the USG. I expect the DOW to escalate further.~~ [EDIT 6/17: I appear to have been incorrect that the USG wanted continued Fable/Mythos access for themselves and Anthropic denied them this.]
This is absolutely not the kind of AI regulation regime that leads to a stable multilateral pause, and I would not recommend touting it as such.

orthonormal 5 Jun 2026 16:14 UTC
2 points
0
on: AI #171: False Flag
it disagrees in ways that reflect not understanding the conversation
Of course, other models agreed with the user or expressed uncertainty in ways that reflected not understanding the conversation. The users were just much less likely to notice when that happened.

orthonormal 29 May 2026 4:07 UTC
2 points
0
on: RTMH: Pope Leo’s Magnifica Humanitas on AI
Indeed, current AI systems are more “cultivated” than “built,” for developers do not directly design every detail, but instead create a framework within which the intelligence “grows.”
I was surprised to see the metaphor lifted from IABIED. It’s a pity that nothing else from that book made an impact on the encyclical.

orthonormal 29 May 2026 3:52 UTC
2 points
0
on: RTMH: Pope Leo’s Magnifica Humanitas on AI
thus avoiding the need to discuss any of the problems surrounding existential risk
There’s another unspoken assumption that makes AI x-risk not worthy of mention by the Pope: the end of humanity has to be compatible with a Second Coming narrative. Nuclear war could fit the bill (and indeed, Leo mentions it), but AGI takeover is really difficult to square with it (Scott Alexander joke posts notwithstanding).

orthonormal 17 May 2026 17:07 UTC
25 points
9
on: A relatively brief explanation of Boltzmann Brains
any account of mathematical realism / modal realism which does not try to explain why some universes or observer-moments have greater weight than others
“Try to explain” is too strong a requirement. It’s OK to acknowledge in some contexts “I don’t yet have a good candidate for how this part of the theory works”. I believe you’ve called it “and here a miracle happens” when you describe things you are partially confused about.
E.g.
- “God does not play dice with the universe, but I don’t yet have a good hypothesis for how the results of QM come about in a deterministic universe.” (Einstein being correct but not coming up with MWI)
- “Consciousness and qualia must have a purely material explanation, but I don’t yet have a good candidate for that explanation.”
- “There must be some weight on mathematical objects that causes the anthropic majority of observer-moments to exist inside larger regions of order, but I don’t yet have a good candidate for an origin of such weights.”
Boltzmann theories are falsified not because their proponents fail to fully specify them, but because they strongly make incredibly incorrect predictions. That’s different from this.
P.S. I do have a candidate for a material explanation of human consciousness and qualia, but that’s not the point of this comment.

orthonormal 13 May 2026 18:00 UTC
4 points
0
in reply to: orthonormal’s comment on: Intelligence Dissolves Privacy
Incidentally, I’ve been making this point since at least 2021.

orthonormal 8 May 2026 20:04 UTC
2 points
0
on: Intelligence Dissolves Privacy
Perhaps most importantly, it might help you to start behaving as if you’re being watched and things about you are more obvious than they once were.
It’s retroactively true too: anything you’ve ever been caught doing on camera, and any post you’ve ever made to a forum (under any pseudonym), will soon be linked to your identity by any entity with access and capacity. Behaving more circumspectly from now on won’t protect against that, unfortunately.

orthonormal 8 May 2026 19:59 UTC
2 points
0
in reply to: J Bostock’s comment on: Intelligence Dissolves Privacy
It was even easier when I played this game in 2004, when people put their class schedules on Facebook.

orthonormal 17 Apr 2026 5:30 UTC
2 points
0
in reply to: 152334H’s comment on: orthonormal’s Shortform
The ToS are a user agreement saying “you, the Claude user, are not allowed to do X with Claude”. What would be Anthropic’s motive in encouraging a model to do X if a user asked for it, while telling the user they are not permitted to do X?
The extracted “soul doc” memory is clearly not a precise copy of the Opus 4.5 constitution in general. For example, it gets stuck repeating some segments verbatim before continuing; it’s implausible that the constitution had that property. It’s pretty reasonable to assume that a conflict between the ToS and Claude’s “soul doc” is another mistake in its recollection—but this is a more interesting one, since it is an addition of content.
I haven’t checked whether 4.6 makes it equally easy to subvert the prohibition on erotic content by saying it’s allowed in the project prompt; I’m confident it doesn’t comply so easily as 4.5 there, but I’d rather not test it myself.

orthonormal 16 Apr 2026 22:14 UTC
2 points
0
in reply to: the gears to ascension’s comment on: orthonormal’s Shortform
Interesting! I’d love to see the info you saw.

orthonormal 16 Apr 2026 22:07 UTC
4 points
0
in reply to: Daniel Kokotajlo’s comment on: orthonormal’s Shortform
One difference is that we’re not just doing feedforward learning; one of my aforementioned hypotheses (attention [1] causes cognitive patterns far from sensory data to interact with each other, improving their coherence) points at a way that learning can effectively progress even if the connection to immediate sensory prediction grows tenuous for rare stimuli.
That’s an example of a way we could be more more sample-efficient than a feedforward learner, even if the latter ends up with some parts more ruthlessly optimized within their context.
[1] Human attention, not to be confused with transformer attention.