I’d love to here are more details about that race, for example:
How big an area or how many people?
What kind of canvassing? Was it door knocking or street?
What data did you collect?
I’m very excited for more people to chime in with their data.
Tom Smith
Karma: 783
The rough math behind it is that an average canvasser in NY-12 seems to have about 3 meaningful conversations per hour, and those meaningful conversations we estimate are worth about 0.1 counterfactual votes. As an example, a meaningful conversation could start with someone who is totally undecided and ends with them leaning somewhat towards Bores over Lasher—maybe it seems like a 70% chance of Bores. Then we’d discount that by half because they may not actually end up voting, and they may be overstating how much their opinion has changed. (Generally canvassers screen a person early on in the conversation for being a registered Democrat in the district, to avoid having ‘meaningful conversations’ with people who can’t actually vote.) That gets to about 0.3 overall, but then we should still probably adjust our effect sizes down somewhat. And again bear in mind most elections are probably not as good as NY-12 for various reasons.
Thanks so much. I fixed it. I’m not sure how that happened honestly.
Thanks for writing this; I imagine it’s a tricky subject to speak on. I broadly agree with the first and last sections of your post, but I have several questions and quibbles with the section on OpenAI’s deal with the Department of War.
You’re placing a lot of faith in the understanding between OpenAI and the DoW:
I feel that too much of the focus has been on the “legalese”, with people parsing every word of the contract excerpts we posted. I do not dispute the importance of the contract, but as Thomas Jefferson said “The execution of the laws is more important than the making of them.” The importance of a contract is a shared understanding between OpenAI and the DoW on what the models will and will not be used to do.
I don’t understand why you think the DoW will act in good faith. Their interactions with Anthropic seem outlandishly, dangerously bad faith. Read this tweet from the DoW’s director and tell me if that sounds like someone you can come to reliable shared understanding with? And more broadly, when you look at the conduct of the current administration, do you believe they will not push boundaries, overreach, and interpret statements in disingenuous ways?
While I think shared understanding is valuable, I think the main point of a contract is to have options for legal redress or enforcement if that shared understanding is violated: when I signed a lease with my landlord, we had a shared understanding that he’d fix the dishwasher if it broke. When he didn’t actually fix the dishwasher, I was very glad I had a contract with some legal remedies.
For this contract to be meaningful, it seems to me like it at a minimum[1] needs to be airtight enough that the DoW won’t be able to weasel out of it in court even when they’re arguing hard and trying to exploit every loophole. As I say in my recent post, “As long as one party to the contract insists that they haven’t given up anything beyond what’s already illegal, and their reading is (by a stretch) consistent with the language in the contract, there will be ambiguity about whether anything more is required.”
This will involve having to wade through some legalese. My recent Less Wrong post has a section where I give some examples of legal language that looks like it does one thing but in fact does another.
If the contract language is never clarified, it will be disproportionately effective at preventing OpenAI from asserting its rights. In the announcement, OpenAI writes “As with any contract, we could terminate it if the counterparty violates the terms.” But will OpenAI be willing to do that if there’s a 50% chance that courts won’t side with them? What about 20%? If OpenAI terminates a contract and then loses in court, they could be forced to pay extremely high costs in damages. Better legal language would help OpenAI win a court battle if the DoW violates the contract.
It might also not be possible for OpenAI to terminate the contract if the government is caught in breach of the shared understanding, unless the contract language makes clear that the terms were violated:
Jessica Tillipman, a legal expert on government procurement law, writes “I’m also curious about OpenAI’s recourse if the govt crosses a red line. In govt contracts, a contractor can’t just terminate for govt breach (w/ limited exception). If this is an OT [Other Transactions, a particular type of procurement] agreement, they may have negotiated broader termination rights, but we don’t know that.”
Overall do you disagree? Maybe you think OpenAI has some other leverage than the courts here I’m not accounting for?
Bear in mind the DoW reportedly wants to use LLMs to conduct mass domestic surveillance and their senior officials have repeatedly made statements to the effect of “We will not let ANY company dictate the terms regarding how we make operational decisions.”
I also worry you’re too optimistic about other parts of this situation as well. For example you mention safeguards:
It allows us to build in our safety stack to ensure the safe operation of the model and our red lines, as well as have our own forward deployed engineers (FDEs) in place. No safety stack can be perfect, but given the “mass” nature of mass surveillance, it does not need to be perfect to prevent it.
On technical safeguards in general: To the extent you rely on technical safeguards with no legal backing, it seems like you are setting yourself up for the DoW to try to ’jailbreak
But overall, quibbling over these kinds of contract details isn’t as important as getting some external party, or at least a large number of employees, the ability to look at the full contract to decide what it does or doesn’t permit. Boaz, did you get to read the full contract? If not, how can you be so confident about what it says or implies when OpenAI leadership has been mistaken about that with regards to this contract a few times before and the base rate for contracts, including lead clauses that substantially undermine or weaken earlier clauses, is really high.
- ^
Ideally the contract would also include enforcement mechanisms to detect breaches of contract and good remedies if there is a breach of contract!
- ^
If you don’t have contractual rights, it’s perfectly legal for the DoW to jailbreak your models. ZDR would prevent you from learning about it, and they wouldn’t tell your forward-deployed engineers.
- ^
It is not clear if the deal OpenAI is negotiating with the DoW provides meaningful red lines against LAWs or domestic mass surveillance.
Edit based on new information: it appears it does not. It seems like “all lawful use” with examples added for clarity. Original comment below.
Reportedly: “OpenAI is pursuing a deal ‘that allows our models to be deployed in classified environments and that fits with our principles. … We would ask for the contract to cover any use except those which are unlawful or unsuited to cloud deployments, such as domestic surveillance and autonomous offensive weapons.’”
Sam Altman seems to be taking the moral high ground here and people have been patting him on the back, but I am unclear on a lot of crucial details, so I’m not ready to pop the champagne just yet.
He says “We have long believed that AI should not be used for mass surveillance or autonomous lethal weapons, and that humans should remain in the loop for high-stakes automated decisions. These are our main red lines” but the deal he’s proposing doesn’t clearly seem to actually enforce that!
One read is they’re planning to fulfill the DoW’s requests for ‘all lawful use’ and their only other restriction is whether they even have the technical capacity to meet the DoW’s requests.
If this interpretation is wrong, it would be great to get clarity on that. I recognize that these kinds of negotiations are tense and that the Pentagon probably wants to save face. But I think it would be premature to congratulate OpenAI before we have actually confirmed that they’re not just caving in and spinning it as though they’re not.
Here are some questions where if I got clear answers that would ameliorate my concerns:
Who determines if these models are “unsuited for cloud deployment”? The DoW or OpenAI? How would they make that determination?
What makes mass domestic surveillance and/or lethal autonomous weapons “unsuitable for cloud deployment”? Is that likely to change?
Who determines if use is “unlawful” in this agreement, OpenAI or the DoW, using the same mechanisms they would use for any other lawful use agreement?
Would this allow types of domestic mass surveillance that Anthropic’s red lines would have ruled out? “Mass surveillance” isn’t a legal term, so when the Department of War says it’s illegal, it’s not obvious what they mean. Some kinds of things I’d consider domestic mass surveillance seem potentially legal, as Anthropic gestured at in their statement. Moreover, the laws here seem pretty fragile and easy to change: the bulk of U.S. foreign intelligence surveillance still operates under an executive order which Reagan signed in 1981, Bush expanded in 2008, and any president can unilaterally amend without a vote in Congress.
How would any restrictions against lethal autonomous weapons and domestic mass surveillance be enforced?
It’s looking like any AI that anyone in the US builds will be used for whatever purpose the government wants and modified to meet the government’s needs.
Anthropic refused to let the Department of War use their models to spy on Americans en masse or autonomously kill people, and for the past week the DoW has been trying to pressure them to change that.
Today the Department of War said “If Anthropic doesn’t let them use their models for any legal purpose the Pentagon wants (“all lawful use”) the Pentagon will either cut ties and declare Anthropic a ‘supply chain risk,’ or invoke the Defense Production Act to force the company to tailor its model to the military’s needs.”[1] The DoW gave Dario until Friday to make a decision.
The former would prevent any company that does business with the DoW from using Anthropic products,[2] something normally reserved for foreign adversaries, and has been threatened for a week. The latter is new, and IMO it’s a big deal.
It would mean no matter what Anthropic does, they can’t control how the DoW uses their models. It’s somewhat ambiguous whether this would be a legal use of the DPA; it’s normally reserved for hardware not software. It’s also not clear to me what it would meant to have Anthropic “tailor its model” to DPA usage.
People at AI companies often assume if they trust the leadership of their company, then they don’t have to worry about egregious misuse. But if they develop the technology, they can’t stop the government form getting their hands on it. And this incident is evidence the government is very willing to take extreme measures to get their hands on it and that they intent to use it for things like spying on Americans en masse.
I expected at some point when AI was very powerful governments would try to nationalize it, but I didn’t expect this kind of action when the technology was this early along in development, when it was very far from posing a decisive strategic advantage.
- ^
It’s important to bear in mind they are probably trying to sound extra scary to pressure Anthropic and other AI developers in this negotiation. Though they also framed this as an ultimatum and didn’t give themselves much room to back down.
- ^
Perhaps from using them at all for anything, perhaps from using them to fulfill that specific contract. The details are murky. Per The Verge:
“This could be implemented in a very narrow sense — or an extremely broad one. ‘I suspect the more logical explanation would be the narrower definition, that Anthropic can’t be used as part of a specific statement of work for the Pentagon,’ said Gertz. ‘But based on some of the reporting and effort to make this seem like a punitive move against Anthropic, it’s worth thinking through both of those scenarios.’ ”
- ^
My claim that Anthropic is the only model the military entrusts for using classified systems is based on the fact that the article I linked says “Anthropic’s Claude is the only AI model currently available in the military’s classified systems” (and this claim has been corroborated by other reporting on the topic that seems to have done original digging). This article goes into more detail.
The government implied that OpenAI, GDM, and xAI will allow their models to be used for mass surveillance of Americans. Are they right?
The Department of War is trying to pressure Anthropic to allow their models to be used “to spy on Americans en masse, or to develop weapons that fire with no human involvement”. Secretary of War Pete Hegseth is reportedly “close” to having the military refuse to do business with any company that doesn’t cut ties with Anthropic. A senior Pentagon official says he wants to “make sure they pay a price for forcing our hand like this.” (Source: Axios)
Right now Claude is the only model that the military entrusts for use in classified systems, but soon they’ll presumably switch to another company if Anthropic doesn’t back down.
The article states
A senior administration official said the Pentagon is confident the other three [OpenAI, Google, and xAI] will agree to the “all lawful use” standard. But a source familiar with those discussions said much is still undecided.”
So it sounds like the government is, as a pressure tactic, implying OpenAI, Google, and xAI will roll over and let their models be used to surveil Americans and autonomously kill people.
Is this true? I assume OpenAI, Google, and xAI employees wouldn’t stand for this. Can OpenAI, Google, and xAI comment on if they will allow their models to be used to surveil Americans en masse or to autonomously kill people without safeguards (esp. measures to ensure they’re not used against Americans)?
This comment strikes me as a bit uninformed. For example, Schlossberg is more of an influencer than a serious candidate. Prediction markets have him hovering at 1-2% right now, which is about as low as a low-liquidity market gets. Folks I talk to on the street seem to very rarely support him. And he is hardly “uncontroversial”. (I suspect that’s why this comment is downvoted.)
Lasher, on the other hand, is a very serious contender and the frontrunner. I more-or-less agree with the prediction markets’s odds on the race (right now: Lasher ~62%, Alex ~38%).
Some thoughts:
Alex has a lot going for him. He’s passed an impressive amount of thoughtful legislation in the NY State Assembly on a wide variety of issues (and has been in office longer than any of his opponents). You can look at all of his bills here. This has earned him the union endorsements, the LGBTQ endorsements, and 13 out of 15 of the endorsements of his fellow assembly members who have weighed in.
I don’t think Lasher is more progressive. Bores and Lasher have very similar policies, but Bores is generally seen as a tad further left (though also note that in NY-12 being further left doesn’t necessarily improve your odds of winning). Which of Lasher’s policies do you think are more progressive?
NY-12 Dem primary voters are unusually educated and politically informed, and even those who aren’t often have a friend community organization that is savvy and to whom they defer. When you dig into the Palantir stuff, I think it looks more reasonable (specifically Alex worked at Palantir under the Obama admin and claimed (even before he was in politics) that he worked on things like closing opioid pill mills, biosecurity, and improving VA staffing. Alex’s version of the sexual harassment allegation is “I then retold a story of something that had happened to me on the job. Someone who didn’t like that retelling had talked to human resources. H.R. had one conversation with me where I shared exactly what had happened, and that was the end of it.)
People are aware there’s a lot of dirt-digging and smear-campaigning going on around this Alex.[1] Many people’s reaction is ~”If this is the worst dirt they can dig up about Alex after all that effort, I’m not that concerned.”
The reality is many people will decide whom to vote for based on if that person has already been representing them and has built ties to their community. Bores is I’m the State Assembly rep for the East Side. Lasher is the State Assembly rep for the West Side. Many people will vote accordingly.
The raw data looks pretty good! Bores seems to be polling well and more or less neck and neck with Lasher.
Note the sexual harassment allegations come from Palantir (the Palantir co-founder funds Leading the Future, the anti-AI-regulation super PAC), were published at a very strategic time for this race, and were published in Bloomberg (Michael Bloomberg has strongly backed Lasher with a $10M super PAC).