I agree with all of this—but also do think that there’s a real aspect here about some of the ideas lying around embedded existing policy constraints that were true both before and after the policy window changed. For example, Saudi Arabia was objectively a far better target for a 9/11-triggered casus belli than Iraq (15 of the 19 hijackers were Saudi citizens, as was bin Laden himself!), but no one had a proposal to invade Saudi Arabia on the shelf because in a pre-fracking United States, invading Saudi Arabia would essentially mean “shatter the US economy into a third Arab Oil Embargo.”
davekasten
Karma: 1,304
I’m kind of confused by why these consequences didn’t hit home earlier.
I’m, I hate to say it, an old man among these parts in many senses; I voted in 2004, and a nontrivial percentage of the Lesswrong crowd wasn’t even alive then, and many more certainly not old enough to remember what it was like. The past is a different country, and 2004 especially so.
First: For whatever reason, it felt really really impossible for Democrats in 2004 to say that they were against the war, or that the administration had lied about WMDs. At the time, the standard reason why was that you’d get blamed for “not supporting the troops.” But with the light of hindsight, I think what was really going on was that we had gone collectively somewhat insane after 9/11 -- we saw mass civilian death on our TV screens happen in real time; the towers collapsing was just a gut punch. We thought for several hours on that day that several tens of thousands of people had died in the Twin Towers, before we learned just how many lives had been saved in the evacuation thanks to the sacrifice of so many emergency responders and ordinary people to get most people out.
And we wanted revenge. We just did. We lied to ourselves about WMDs and theories of regime change and democracy promotion, but the honest answer was that we’d missed getting bin Laden in Afghanistan (and the early days of that were actually looking quite good!), we already hated Saddam Hussein (who, to be clear, was a monstrous dictator), and we couldn’t invade the Saudis without collapsing our own economy. As Thomas Friedman put it, the message to the Arab world was “Suck on this.” And then we invaded Iraq, and collapsed their army so quickly and toppled their country in a month. And things didn’t start getting bad for months after, and things didn’t get truly awful until Bush’s second term. Heck, the Second Battle for Fallujah only started in November 2004.
And so, in late summer 2004, telling the American people that you didn’t support the people who were fighting the war we’d chosen to fight, the war that was supposed to get us vengeance and make us feel safe again—it was just not possible. You weren’t able to point to that much evidence that the war itself was a fundamentally bad idea, other than that some Europeans were mad at us, and we were fucking tired of listening to Europe. (Yes, I know this makes no sense, they were fighting and dying alongside us in Afghanistan. We were insane.)
Second: Kerry very nearly won—indeed, early on in election night 2004, it looked like he was going to! That’s part of why him losing was such a body blow to the Dems and, frankly, part of what opened up a lane for Obama in 2008. Perhaps part of why he ran it so close was that he avoided taking a stronger stance, honestly.
I mean, two points:
1. We all work too many hours, working 70 hours a week persistently is definitely too many to maximize output. You get dumb fast after hour 40 and dive into negative productivity. There’s a robust organizational psych literature on this, I’m given to understand, that we all choose to ignore, because the first ~12 weeks or so, you can push beyond and get more done, but then it backfires.2. You’re literally saying statements that I used to say before burning out, and that the average consultant or banker says as part of their path to burnout. And we cannot afford to lose either of you to burnout, especially not right now.
If you’re taking a full 4 weeks, great. 2 weeks a year is definitely not enough at a 70 hours a week pace, based on the observed long term health patterns of everyone I’ve known who works that pace for a long time. I’m willing to assert that you working 48/50ths of the hours a year you’d work otherwise is worth it, assuming fairly trivial speedups in productivity of literally just over 4% from being more refreshed, getting new perspectives from downing tools, etc.
I’d like to strongly assert that you’d want your design spec to be multiplayer from the start so that you can have virtually any arbitrary mix of LLMs and people. You’ll probably want this later and there are likely to be some design decisions that you’ll make wrong if you assume there’s never more than one person
I would strongly, strongly argue that essentially “take all your vacation” is a strategy that would lead to more impact for you on your goals, almost regardless of what they are.
Humans need rest, and humans like the folks on LW tend not to take enough.
“We don’t want it to be the case that models can be convinced to blackmail people just by putting them in a situation that the predictor thinks is fictional!”
This is interesting! I guess that in, some sense, means that you see certain ways in which even a future Claude N+1 won’t be a truly general intelligence?
Sincere request that Anthropic work with Reed to bring Person of Interest back to Netflix and give it the Suits promotional push this summer. (I am not kidding. I mean this, actually.)
I would note that this is, indeed, a very common move done in DC. I would also note that many of these copies end up in, e.g., Little Free Libraries and at the Goodwill. (For example, I currently downstairs have a copy of the President of Microsoft’s Board’s book with literally still the letter inside saying “Dear Congressman XYZ, I hope you enjoy my book...”)
I am not opposed to MIRI doing this, but just want to flag that this is a regular move in DC. (Which might mean you should absolutely do it since it has survivorship bias as a good lindy idea! Just saying it ain’t, like, a brand new strat.)
We’re hiring at ControlAI for folks who walk to work on UK and US policy advocacy. Come talk to Congress and Parliament and stop risks from unsafe superintelligences! controlai.com/careers
(Admins: I don’t tend to see many folks posting this sort of thing here, so feel free to nuke this post if not the sort of content you’re going for. But given audience here, figured might be of interest)
I think I am too much inside the DC policy world to understand why this is seen as a gaffe, really. Can you unpack why it’s seen as a gaffe to them? In the DC world, by contrast, “yes, of course, this is a major national security threat, and no you of course could never use military capabilities to address it,” would be a gaffe.
I particularly appreciated its coverage of explicitly including conventional ballistic escalation as part of a sabotage strategy for datacenters
One thing I find very confusing about existing gaps between the AI policy community and the national security community is that natsec policymakers have already explicitly said that kinetic (i.e., blowing things up) responses are acceptable for cyberattacks under some circumstances, while the AI policy community seems to somehow unconsciously rule those sorts of responses out of the policy window. (To be clear: any day that American servicemembers go into combat is a bad day, I don’t think we should choose such approaches lightly.)
I think a lot of this boils down to the fact that Sam Vimes is a copper, and sees poverty lead to precarity, and precarity lead to Bad Things Happening In Bad Neighborhoods. The most salient fact about Lady Sybil is that she never has to worry, never is on the rattling edge; she’s always got more stuff, new stuff, old stuff, good stuff. Vimes (at that point in the Discworld series) isn’t especially financially sophisticated, so he narrows it down to the piece he understands best, and builds a theory off of that.
You can definitely meet your own district’s staff locally (e.g., if you’re in Berkeley, Congresswoman Simon has an office in Oakland, Senator Padilla has an office in SF, and Senator Schiff’s offices look not to be finalized yet but undoubtedly will include a Bay Area Office).
You can also meet most Congressional offices’ staff via Zoom or phone (though some offices strongly prefer in-person meetings).
There is also indeed a meaningful rationalist presence in DC, though opinions vary as to whether the enclave is in Adams Morgan-Columbia Heights, Northern Virginia, or Silver Spring.*
*This trichotomy is funny, but hard to culturally translate unless you want a 15,000 word thesis on DC-area housing and federal office building policy since 1945 and its related cultural signifiers. Just...just trust me on this.
The elites do want you to know it: you can just email a Congressional office and get a meeting
I think on net, there are relatively fewer risks related to getting governments more AGI-pilled vs. them continuing on their current course; governments are broadly AI-pilled even if not AGI/ASI-pilled and are doing most of the accelerating actions an AGI-accelerator would want.
The Trump administration (or, more specifically, the White House Office of Science and Technology Policy, but they are in the lead on most AI policy, it seems), are asking for comment on what their AI Action Plan should include. Literally anyone can comment on it. You should consider commenting on it, comments are due Saturday at 8:59pm PT/11:59pm ET via an email address. These comments will actually be read, and a large number of comments on an issue usually does influence any White House’s policy. I encourage you to submit comments!
regulations.gov/document/NSF_FRDOC_0001-3479… (Note that all submissions are public and will be published)
(Disclosure: I am working on a submission for this for my dayjob but this particular post is in my personal capacity)
(Edit note: I originally said this was due Friday; I cannot read a calendar, it is in fact due 24 hours later. Consider this a refund that we have all received for being so good at remembering the planning fallacy all these years.)
I think there’s at least one missing one, “You wake up one morning and find out that a private equity firm has bought up a company everyone knows the name of, fired 90% of the workers, and says they can replace them with AI.”
This essay earns a read for the line, “It would be difficult to find a policymaker in DC who isn’t happy to share a heresy or two with you, a person they’ve just met” alone.
I would amplify to suggest that while many things are outside the Overton Window, policymakers are also aware of the concept of slowly moving the Overton Window, and if you explicitly admit you’re doing that, they’re usually on board (see, e.g., the conservative legal movement, the renewable energy movement, etc.). It’s mostly only if you don’t realize you’re proposing that that you trigger a dismissive response.
Ok, so it seems clear that we are, for better or worse, likely going to try to get AGI to do our alignment homework.
Who has thought through all the other homework we might give AGI that is as good of an idea, assuming a model that isn’t an instant-game-over for us? E.G., I remember @Buck rattling off a list of other ideas that he had in his The Curve talk, but I feel like I haven’t seen the list of, e.g., “here are all the ways I would like to run an automated counterintelligence sweep of my organization” ideas.
(Yes, obviously, if the AI is sneakily misaligned, you’re just dead because it will trick you into firing all your researchers, etc.; this is written in a “playing to your outs” mentality, not an “I endorse this as a good plan” mentality.)
I think this is somewhat true, but also think in Washington it’s also about becoming known as “someone to go talk to about this” whether or not they’re your ally. Being helpful and genial and hosting good happy hours is surprisingly influential.