I work at the Alignment Research Center (ARC). I write a blog on stuff I’m interested in (such as math, philosophy, puzzles, statistics, and elections): https://ericneyman.wordpress.com/
Eric Neyman
Karma: 6,390
Thinking about math is counterproductive for me, but I’ve found word puzzles to be good for this. For example: pick two four-letter words and try to find a word ladder that connects them (ie, a sequence of words that only differ by one letter).
Mergers and acquisitions are rare in the nonprofit world, I believe, which seems like a better references class for EA than corporations.
Interesting, I update a lot on you saying that you’d take best against “more than 10”—though I’m surprised.
I would guess that, if one were to look at all recent times that someone has finished reading a post for which there was at least one comment and at least one “mentioned in” post:
Slightly over half of the time, the person then checked out the comments.
The person clicked on a “Mentioned In” post less than 5% of the time.
Curious which of these is/are the crux.
Not a solution, but I just wanted to voice that I’d be sad to see you go, in part because I think there’s a decent chance that your “hobby horse” is very importantly right.
As a data point, I use LW a lot and wasn’t aware of the “Mentioned In” section. And I think the fact that it’s a different level of visibility than being in the comments actually drops readership by a quite large factor (I’d guess more than 10).
If I’m right, then—insofar as Wei is unhappy about the fact that he can’t respond in a way that gets lots of views—I don’t think the “Mentioned In” section adequately addresses this.
(This comment isn’t meant as an endorsement of the comment from Wei that you’re replying to.)
Note that (I think?) most conjectures are true, though! So you shouldn’t expect 50⁄50 by default.
I probably agree that if you told this sentence to EAs, then they’d be confused or think that things have gone wrong. But like, it’s usually the case that you can give partial information about a situation that leads people to be confused?
I think that if you gave the additional information of “Oh by the way, advanced AI seems imminent, and we are absolutely not on track to handle the situation well, but some candidates with great ideas and a good understanding of AI policy are running for office and could move the needle”, then they would no longer be confused.
I’m not familiar with New York politics in particular, but—at least on the federal level—co-sponsorship doesn’t tell you very much beyond “this person supports the bill”. The RAISE Act had 34 co-sponsors (Lasher among them). Alex, on the other hand, was the driving force behind the bill, helped write the bill, and expended a really large amount of political capital getting it through.
I wouldn’t say that the bridge-burning effect is zero, but I think that raising Alex’s a priori win probability from 5% to 30% was dramatically larger (like, by more than a factor of 100) than the bridge-burning effect, given that Lasher likely won’t be one of the leading representatives on the issue.
There are many people in Congress who favor a data center moratorium. However, none of them have AI as their #1 issue, and this is also true of Lasher. I expect Lasher to be a good member of Congress, but I don’t think Congress will be meaningfully different in its orientation to AI issues because he’s there.
Meanwhile, I think Congress would have been meaningfully different on AI issues if Bores had won. Bores has made AI his #1 issue, and I think he would have done all he could to pursue policies that try to mitigate AI x-risk. He would have staked a lot of political capital on that, as he did with the RAISE Act.
I expect to write a longer reflection, but my position is that this was a really good bet for us to have made, even if it didn’t pan out (and indeed, in my original post, I assigned only a 20% chance to it panning out).
(I also think that the positive effect on the narrative around LTF if Bores had won would have been a very small fraction of the direct positive effect of having Bores in the US government.)
I don’t, sorry! Although, for what it’s worth, in the two conversations I’ve had with him about risks from AI, China didn’t come up.
Edited to add: also, for what it’s worth (and apologies for venturing into explicitly partisan politics), I am personally substantially more worried about AI-enabled authoritarianism under Trump than under most other possible presidents (though my worry definitely doesn’t go away entirely).
I don’t speak for Bores, but I think Eli is correct.
Thanks for posting this, and it’s something I’ve been thinking about, too. I think that running for office is really rough, and—pretty unfortunately—there can be a trade-off between sticking to your guns on your beliefs and winning elections. I’m not sure what to do about this and how I would advise politicians to navigate it.
I think I straightforwardly disagree with this part:
Alex Bores seems like a very mildly below-average integrity politician
I’m not sure exactly where my view (that Alex is an above-average integrity politician) comes from, but here are some pieces of evidence:
During his time in the Assembly, he has sometimes been one of the only Democrats voting against tech regulation bills that didn’t make sense. This is despite those votes being politically inexpedient for him. See here for some examples; my understanding is that A4550 is an example of straightforwadly bad regulation.
I believe that the amount of political capital he staked on fighting for a strong version of the RAISE Act in December is much larger than the most politically opportune amount of capital for him to have staked.
More generally, people who have worked closely with him say that he cares a ton about the substance of bills he fights for, rather than accepting more superficial wins that are politically expedient.
I also want to push back on this part--
If, based on the dozens of people in the AI Safety community writing glorious exhortations of him, you ended up with the impression that when Alex says something you can take it remotely as seriously as posts around here, please update.
--to which I reacted with “agree denotationally but object connotationally”. Here’s what I believe, and I’m curious if you disagree with: I think we have reasonably strong evidence that Bores’s primary concern on AI issues is catastrophic and existential risks.[1]
I have two main pieces of evidence for this:
The people who have worked most closely with him on bills—who, by the way, have dealt with many politicians—believe it to be true.
His defense of the RAISE Act on the State House floor stressed catastrophic risk in a way that actually pushed back on concerns about more immediate harms.
This doesn’t address all of your critiques of Bores, and I’m pretty sympathetic to your critique of his social media presence.
- ^
My guess is that his top two concerns are bio misuse risk and AI-enabled authoritarianism; I believe yours is AI takeover, so I’m not saying that you and Bores are precisely aligned.
A heuristic I’ve heard used is that spending on advertising in primary elections in inexpensive media markets has an effect of roughly $100/net vote. On the other hand, Berkeley is an expensive area to run ads, so it should be higher.
(And this should be treated as a primary election, because it’s a Democrat vs. Democrat contest.)
Sometimes we see races where one side starts spending large amounts all of a sudden and the polls shift; this lets us get an okay-but-not-great measurement of the cost-effectiveness of ads. See eg here (search “Iowa”) for an example—though admittedly that was spending by a super PAC, which is generally somewhat less effective than campaign spending, so you should divide those cost estimates, maybe by a factor of 1.5 or so.
My guess would be something like: starts at $200/vote and then gradually falls off as diminishing marginal returns kick in.
But in terms of my overall appraisal of the situation, I’m operating under a different heuristic: incumbents are very difficult to beat unless they are unpopular. And, as far as I know, Lateefah Simon is perfectly popular.
Jamie raised $14K so far and spent 10k on the primary, winning 28K votes (!!), meaning getting a vote for 36cts, which seems fairly efficient.
For figuring out spending effectiveness, the question is how many votes she would have gotten if she hadn’t spend any money on the primary. My guess is, perhaps, a couple hundred fewer votes.
Thanks for writing this, Jeff!
If anyone is interested in receiving political donation recommendations, feel free to fill out this form!
Do we know that he didn’t just change his mind?
I think voters might not trust that the politicians will in fact vote in accordance with their constituents’ beliefs. It’s hard to credibly commit to that.
Campaign update: With 56% of the vote in, AI safety champion Scott Wiener and Connie Chan (both Democrats) have advanced to the November election. Wiener got 41.3% of the vote, compared to Chan’s 28.6%. This is roughly what I expected.
Wiener is favored to win in November, but it’s not a done deal, for a few reasons:
Voters for the third-place finisher, Saikat Chakrabarti, might go disproportionately to Chan over Wiener: I think Chakrabarti is more ideologically similar to Chan overall, though it’s not totally clear.
Republicans really don’t like Wiener, and might prefer Chan over Wiener.
Nancy Pelosi, who endorsed Chan, will probably use what power she has to try to further boost Chan.
Wiener is about 70% to win per prediction markets, and that sounds about right to me. (Maybe it’s a tad low; my intuition is more like 75%.)
The change in verbiage from “slowdown” to “pacing” has been consequential to public discussion about the speed of AI progress. But this change isn’t as new as it may seem. On December 22, 2022, Katja Grace wrote a post advocating for slowing down AI:
What I didn’t realize until now is that she followed it up just 11 days later with a post advocating for “pacing”:
Thanks to Katja for pioneering this consequential change!