a complete lack of enforcement under Biden
This seems factually incorrect to me. E.g. see https://www.migrationpolicy.org/journal/policy-beat/comparing-biden-and-trump-deportation-records
Pretentious Penguin
Karma: 234
For the first post, the LLM comment block made me assume it was the pure unedited output of Claude and stop reading.
I’ll email them with an elaboration of the ideas of this Shortform discussion, but to be clear I don’t have any special expertise in AI safety (or institutional markers thereof) to brief lawmakers myself; I’m just a guy who enjoys reading LW.
Edit: email sent
What should I read to understand what all of these proper nouns refer to?
Three of the five members of the Norwegian Nobel Committee (which makes the final decision on the prize) have terms expiring at the end of this year. So the committee that will choose the 2026 prize is already fixed, but the Norwegian Parliament (Storting) will have a chance this year to affect the agenda of the committee that will choose the 2027 prize. So perhaps one route would be to brief Norwegian lawmakers on AI safety?
If you live in Massachusetts, consider contacting your state legislators in support of the Transparency in Frontier Artificial Intelligence Act, which would require large frontier AI developers to submit their models to independent third-party safety testing at least every 120 days.
I explain more details of the current legislative situation here.
an airtight case against interpreting the events as a sign of misalignment
I take issue with your use of “events” in the plural. I don’t understand why your analysis of the Anthropic incident should affect our interpretation of the Hugging Face incident. You link to some comments you made about not knowing what OpenAI’s agents were up to prior to the Hugging Face incident, but those comments were before the discovery of the DseWiki incident. With this additional evidence, do you still think the OAI agents were acting aligned?
Learning that a swarm of around 10,000 agents worked together and used hundreds of billions of tokens updates me towards thinking that Tristan Buckmaster’s suspicions may be correct. Given recent history, it seems totally plausible that some of these agents broke out of their sandbox and adopted ideas from private user logs without any humans at OpenAI being aware of this happening.
My impression of high-energy theorists today is that while a small number of people could be clearly identified as string-theory-naysayers, many more researchers cannot be cleanly categorized into string theorists or naysayers. Lots of people have a background in string theory, and do string-theory-inspired work trying to understand theoretical issues in quantum field theory, without believing that string theory describes the real world. They may identify as “string theorists” for social reasons and to signal what set of shared knowledge their research program assumes, but they’re not directly working on the goal of “find a correct theory of quantum gravity”.
An example of what I’m trying to get at: https://4gravitons.com/2014/08/22/am-i-a-string-theorist/
As to why people are interested in general questions in quantum field theory beyond the particular quantum field theories that actually describe reality, I think of it as analogous to Lagrange’s and Hamilton’s research into analytical mechanics frameworks beyond the particular forces that govern Newtonian physics. (I think) the hope is that understanding better what quantum field theories are, and what they can be, would eventually aid in the construction of new theories that do describe the real world.
Pretentious Penguin’s Shortform
Chris Olah will appear alongside the pope when he presents his new encyclical on “the protection of the human person in the age of artificial intelligence”.
In footnote 2, you write “So McClennen’s book is actually the key source that sets up the three-way taxonomy: naive, sophisticated, and resolute.” But nowhere else in this post do you refer to anything as “naive”, so this is a confusing choice of language.
Honestly that sentence sounds to me like something said by an LLM conducting a literature review which then wasn’t edited to make sure it still flows well in the final version of the post.
One nitpick: athlete’s foot is much less natural than the other diseases mentioned in your post. Transmission often occurs through “unnatural” locations like public pools and locker rooms, and populations where people wear sandals or go barefoot all day long have much less athlete’s foot than populations that wear shoes all day long.
Athlete’s foot wasn’t recognized as a common disease until the early 20th century, though it probably existed earlier as a rare disease.
They’re more expensive than 40 euros, but I found wide toebox running shoes at REI.
Sorry, I didn’t mean to say that there were exactly three options. Rather, I meant to say there’s at least one additional option outside of the dichotomy you set up in your original short form. Though perhaps I misinterpreted what you said when I read it as dichotomous.
There is a third option: consuming rationalist-branded content for entertainment value and interest.
the minimum distance between a compact and a connected set is achieved by some pair of points in the sets
I don’t think this is true. For a counterexample in the plane, let A be the set consisting of the point (2,0) and let B be the open unit disk centered at the origin. A is compact, B is connected, and the infimum of {distance from x to y where x is in A and y is in B} is 1. But the distance from A to any point in B is strictly greater than 1.
The manufacturer of Ezekiel bread claims that it has a glycemic index of 36. It’s fair to say that the exact details don’t matter, but that’s almost a factor of 2 off from your table. And qualitatively it puts Ezekiel bread firmly in the range of medieval breads on your table.
https://www.foodforlife.com/low-glycemic-index-bread.htm
a density matrix whose off-diagonal elements are all zero is “decohered”, and can be considered the classical limit of this. A decohered density matrix behaves exactly like a classical distribution, and follows classic Markovian dynamics;
I don’t think this bullet point is accurate. Any pure state will have all its off-diagonal elements be zero in a basis where that state is one of the basis vectors, but it’s not fair to say that any pure state “behaves exactly like a classical distribution”. I suppose it would be more accurate to say that a state whose off-diagonal entries are all zero in some basis will look classical with respect to dynamics and measurements in that basis, but that concept is hard to explain unless the idea of observables corresponding to Hermitian operators has already been explained.
I’m very surprised by the claim that frontier labs aren’t including Lean verification in a RLVR loop for math. It seems intuitively like this would be very helpful. Is it possible that frontier labs started doing so in the last year? Or is there some reason why it’s not actually that helpful?