Idle musing: should we all be writing a Claude Constitution-esque set of posts about our hopes for how AIs help humans around the dangerous moments in takeoff, in hopes that this influences the models advise not only us, but people who are coming into these issues fresher than us?
(Yes, I know that from the outside I have exponentially less influence on model behavior than Anthropic, and for MIRI-like reasons maybe this doesn’t go well at all. But, you know, play to all of your outs.)
davekasten
Karma: 2,026
I would like, for obvious self-interested reasons, discussion of public policy ideas you think people should be pushing more on than they currently are.
It was a mild positive update for me that an Anthropic employee, in this case Zac, was able to clarify this.
I worry, a lot, that the true gloss on the American Way of War is, roughly the meme of “every Pacific naval encounter from late 1943 onward is like the IJN Golden Kirin, Glorious Harbinger of Eternal Imperial Dawn versus six identical copies of the USS We Built This Yesterday supplied by a ship that does nothing but make birthday cakes for the other ships.”
Or, put more generally, we show up in the 4th quarter with a shit ton of gratuitously over-the-top production of every possibly-vaguely-good idea, and manage to eke out a win. See, e.g., the Civil War, WW1, WW2 (as above), Korea (kind of, long story), the Gulf War (after we fucked up the pre-war diplomacy), and the post-Surge Iraq War.
”There is no pause, so we just gotta Get a Lot of Alignment Research Done Real REAL Fast” is plausibly the real world we end up in, and I think we should have more folks optimizing for it beyond Redwood (and Anthropic???), even as terrifying as it feels.
I also had this question and couldn’t find an obvious answer. I recognize that to some degree this might be proprietary, but this feels like a pretty obvious comms question. It doesn’t negatively impact my opinion of Zac or Drake if they’re unable to answer given their confidentiality obligations, but I would ask them to relay back internally that not indicating one way or another looks extremely weird from a comms perspective.
It’s like Ford announcing that they’ve added airbags to their car designs. Which designs, the ones available for purchase now, or future year models? Oh, you know, just...car designs.
This is awesome! It broadly aligns with my understanding of the situation, although it does miss some folks that are known to care a bunch about this from their public statements. Downloading the JSON to take a deeper look!
Strong upvoting the underlying post for Doing The Thing.
So far the LLMs really want to procrastinate on this task in normal chat windows because it’s tooooo many queries. This is gonna have to be a Claude Code thing.
Yeah, I think one thing I’ve added to my (too long!) to-do list is to ask the LLMs, and then pay a researcher to find any other examples of folks like this that we missed.
I just reread https://www.lesswrong.com/posts/CoZhXrhpQxpy9xw9y/where-i-agree-and-disagree-with-eliezer by Paul Christiano from 2022 for somewhat random reasons[1] and wow is this a fascinating historical snapshot document, especially in the comment section.
Many of the main characters in AI from 2022 to 2025 swing by and say, essentially, “hello! I would like to foreshadow my character arc for the next 3 years!”
- ^
Too many open tabs, need to clean up or computer no do videoconference good
- ^
Random note: Congressman Brad Sherman just held up If Anyone Builds It, Everyone Dies in a Congressional hearing and recommended it, saying (rough transcript, might be slight paraphrase): “they’re [the AI companies] not really focused on the issue raised by this book, which I recommend, but the title tells it all, If Anyone Builds It Everyone Dies”
I think this is a clear and unambiguous example of the theory of change of the book having at least one success—being an object that can literally be held and pointed to by someone in power.
I get what you’re saying here (I do live in DC) but I generally think that the returns to “just being a simpler person” (simple in the complexity sense, not the euphemism-for-dumb sense) are deeply underrated.
One of my biggest lessons from being a bartender and waiter in DC is that genuinely prioritizing giving a guest the best answer on which wine to order, even if it resulted in a lower price point for that bottle, generally resulted in a higher total tip amount at the end of the night. People trust you more, at least in American society, if you’re just, like, trying to be honest and give your true best guesses.
This generalized to a hilarious degree to working in management consulting—I assert that a huge reason of why McKinsey is higher prestige than Booz Allen (I worked at both places) is that Booz Allen culture encourages a lot more self-serving behavior, and McKinsey culture encourages serving the best interests for your client, and this affects how people relate to you in a pretty fundamental way.
Something I believe:
The reason that society isn’t currently freaking out about AI taking artists’ jobs is mainly that we’ve historically thought of artists’ jobs as inherently precarious, and so a new report of them being precarious for new reasons doesn’t surprise anyone. The moment it takes a “so stable your mom wants you to study it in school” job, that all will change for the stable folks, but not the artists, unfortunately. After all, they’re just artists...
(If you say that this means that I don’t care about artists or their financial challenges, you’re wrong. This sucks, I’m predicting, not endorsing, a likely scenario.)
Yup, this is very much part of it. But overall, it’s the tightening of a thousand screws rather than One Weird Trick.
In the House, in addition to the tradition changes that MondSemmel mentions, successive speakers over time have modified the formal House Rules iteratively to (generally) consolidate more power under the Speaker in a bunch of bureaucratically technical but important ways. (E.G., the Federalist Society on the right argues that Gingrich’s decision to cut committee staff sizes and impose term limits for committee chairs nerfed the power of committees relative to the Speaker https://fedsoc.org/commentary/fedsoc-blog/navigating-the-rules-of-the-people-s-house). Note that some view the current Speaker as intentionally choosing to begin to reverse this trend as part of commitments he voluntarily made to his side of the aisle to somewhat empower individual Members, but this is seen even by his biggest fans as only a first step, and certainly not universally agreed upon even within his own Party.
In the Senate, similar trends are combined with a slow but fairly steady erosion of the scope of the filibuster (as well as the maximization of procedural hacks around the filibuster, such as how to use reconciliation), which de facto increases the power of the Senate Majority Leader in closely-divided Senates (most have been for the past 20 years). But this is not as far along as in the House, and various Senate Majority Leaders have approached this question differently, even within the same party, so bright line conclusions are harder to draw here. (E.G., Reid and Schumer had somewhat different approaches from each other, as do McConnell and Thune)
This is a good summary by Vox, which in turn points to a bunch of deeper writeups if helpful https://archive.is/o6qj8
Congressmembers do have power, but it’s...weirdly distributed at the veto points? There is some horse trading, but it’s maximally in the “you gotta trust me it’s there” black-box aspects of the system (e.g., the famously opaque negotiation inside the NDAA process).
But power has definitely drained away from individual members; the standard account is roughly that a combo of 0) Increased transparency into Congress via things like C-SPAN and cable news generally, 1) leadership of committees and the parties consolidating more role-based power (especially in the House), 2) reduction in earmark-like things that enabled side deals to get bigger deals done, and 3) increased party polarization due to nationalization of party identities (measured through such things as DW-NOMINATE scores) were the key drivers,
(Looks at half completed pitch deck titled “Starcology: A Lighthaven Managed Residence” and hits delete)
Obvious implication is obvious: Lighthaven should seriously consider ways to capture a share of value form “I have a flow of people moving to the Bay Area”. (Dumb version: find a realtor adjacent to this community, capture a share of referral fees. Other more sophisticated options are left as an exercise to the reader.)
I’m not making a one-time donation to Lighthaven, because I already have a recurring donation set up to Lighthaven. I encourage doing this as a general best practice for those donating ordinary USD or similarly boring funds, especially if your donation approach isn’t set up for a lump-sum approach.
While obviously less valuable in net present value terms than donating the full value upfront, having a predictable recurring fundraising flow from donors is quite useful for nonprofits.
Indeed, the traffic for AI 2027 in 2025 has been similar to the traffic for all of LessWrong in 2025, with about 5M unique users and 10M page-views, whereas LessWrong had 4.2M unique users and 22M page-views.
While my inner consultant loves this approach to impact sizing, I think another approach is to ask how much various political efforts would pay to get the Vice President (which on rough historical odds has ~50% chance of being the next President, in addition to his current powers) to read and publicly say that he’d read a document and had a positive opinion of it.
It is, uh, more like the think tank number than the paid ads number.
Who’s behind Pull the Plug? I don’t see any details about it on their website.