Running Lightcone Infrastructure, which runs LessWrong and Lighthaven.space. You can reach me at habryka@lesswrong.com.
(I have signed no contracts or agreements whose existence I cannot mention, which I am mentioning here as a canary)
habryka
Karma: 61,528
I mean, this is super hard to tell! The primary thing I would want to communicate is “there are costly signals someone can send of being high-integrity, which for example, Scott Alexander and Kelsey and many others in the extended AI Safety space have sent, and he has not done so, and there are a few mild flags that make me think he is not substantially better than other politicians”.
I’ve read a lot of books about politics, talked to a few congress people, have followed a few of them on social media, looked into local politics campaigns in more detail, etc.
Like, I am not a deep scholar of politics, but I try to generally be informed about politics and politicians, with a particular interest in understanding people’s integrity.
That said, this topic is super tricky to judge and I haven’t done a massive deep dive into Bores or any other politician, and I am far from confident.
I am so confused. But sure, here are some examples:
The other day we had an event at Lighthaven that was louder later into the night than usual. I knew our neighbor was going to be bothered by it. So I went to them and said “hey, I am really sorry, I know we are louder until later into the night, and I know this is bothering you. Is there any way I can make you whole? For example, I would happily pay you $200 for the inconvenience”. She thought a bit about it, then seemed to agree that $200 would make her whole.
Like, most of the time when I apologize, the thing we do afterwards is not for me to change my behavior in a way that avoids the harm, it’s to use any of the other much more fungible resources that I have to make the other person whole. Usually that’s just social capital. “Hey, sorry about that, I owe you one”, is a totally normal sentence to say.
In group house contexts being like “hey sorry about it repeatedly falling to you to take out the trash because you are most bothered by it piling up, what’s your happy price for dealing with the inconvenience?” is the kind of thing I say like every second house meeting or so.
FWIW Alex Bores seems like a very mildly below-average integrity politician, having talked to him once and having followed his campaign and social media presence. He seems to say things he doesn’t believe somewhat more often than other politicians, but not much so, and he gives me some amount of “naive-consequentialist EA” vibes that make me think he is higher variance on this dimension than others. He does seem to really care about the AI Safety thing, he really appears to be targeted by a ton of very aggressive attack ads funded by AI capability companies, and it seems really pretty important to counteract that, all of which make me think it’s overall good to vote for him (and to do high-integrity campaigning for him).
His behavior on social media and speeches is very far below the integrity standards for leaders in the AI safety community, and if he behaved similarly here as he behaves in politics, I would advocate strongly against empowering him. Of course politics is a different playing field, and as I said, he seems roughly average for that domain. But if, as a result of people in the AI Safety community writing many glorious exhortations of him, you ended up with the impression that when Alex says something you can take it remotely as seriously as posts around here, please update.
Most of his social media posts and campaign actions appear to be things he doesn’t believe and is choosing to say for political reasons, including on AI topics (on which he happily conflates existential risks with things like trying to blame AI systems for triggering suicides in teenagers). I tentatively think this is OK given the standards in politics, and you ultimately have to vote for someone, but it’s also not great. IMO advocating for regulating AI on the basis of spurious harms like this is bad, will eventually backfire, and should be modeled as a cost, not as a benefit, from the perspective of predicting how useful his actions will be on AI safety topics. If you heard about him making “AI safety” his topic, you should remember he is mostly talking about that kind of safety (though he has mentioned existential risk as well, and I believe takes them genuinely seriously).
He has also proposed some partisan regulation that seems likely unconstitutional to me in a way that I think is substantially more harmful than what I’ve seen most other politicians propose (and would non-trivially escalate the existing intense conflict in US partisan politics).
He has a technical background and congress is very sorely lacking people with technical backgrounds. He has proposed some bad bills, and some quite good bills. Outside of AI safety I am not that impressed with his bill track record, but I haven’t evaluated it that closely. I find myself less excited than others about the RAISE act so I assign less credit for that, but he was a major champion of it and it does seem good on-net. I have not seen him write or say anything that demonstrates a deep understanding of AI safety.
I think it’s probably good to vote for him. I think people making uncaveated endorsements of him in a way that overstate his integrity, or speak with far more confidence about his character than is warranted, are causing great harm. By my lights, feel free to go around saying “vote for Bores, he seems competent, cares about AI Safety, and is about as sketch as the average politician”, but dropping opening sentences like “Alex Bores is the best politician of a generation”, given our current level of knowledge and understanding, seems extremely ill-advised to me.
To be clear, many people have been doing fine here. For example I have no objection of this kind to Eric Neyman’s original post on Bores’, though I do think not trying to evaluate Bores’ integrity was a pretty bad mistake and has predictably resulted in other people thinking about this badly down the line, but it’s not Eric’s job to evaluate every facet of a politician at all times and other people could have filled this niche, and he didn’t overstate his integrity either.
When someone wrongs me, it seems like the smallest ask I could reasonably make in exchange for my forgiveness is that they not do that again (or more generally, update their policy such that they’re less likely to do it again). Not to ask to be “made whole”—for the past cannot be changed—but simply that they do better in the future, which can.
If they refuse, saying, “That’s sad, but I will not stop doing the thing that hurt you. Yet I will take this cost on my ledger. I’m sorry. That’s on me. Think of me as owing you a small something you can cash out another time,” I have to admit I’m skeptical. If I can’t ask not to be hurt again, what can I ask for? Money? Chocolate? Their car?
I don’t understand these paragraphs. Yes, sometimes someone does something that provides them more benefit than it caused you harm. They apologize to you for it. You ask for enough to make you whole. Could be money, chocolate, their car, or some social capital.
Somehow these paragraphs imply that is impossible? I don’t get it. Of course it’s possible. Indeed it’s the default as far as I know!
I think up until very recently funding Lightcone would have been a good bet at your scale (though you should of course be appropriately skeptical of me saying this). Funding projects doing good object-level work seems good.
Before I think about concrete recommendations, by “alignment researchers” do you mean people doing technical alignment work, or eval work, or do you include technical governance work like MIRI’s technical governance team? Or are you just using it as a proxy for work trying to make alignment go well in some form or another?
I think it’s gotten worse over time. I should have said “I don’t think MATS or the labs (or the funders) care much if someone actually cares”. I agree the caring isn’t zero.
This also seems right to me, but I don’t think MATS or the labs (or the funders) care if someone actually cares. They seem happy to tell themselves stories that they are making things better by getting people into the labs (or lab-adjacent orgs) even if they really don’t seem very safety motivated.
IMO the correct equilibrium in order to minimize costs from muggers and increase incentives is not for people who get mugged to defend themselves, but to prosecute those muggers with people who specialize in violent altercations (i.e. the police). A really important part of decision theory and “prisoner’s dilemma” like arguments for this kind of stuff is to think about what the actually achievable and good equilibrium is, and “everyone gets good at self-defense” is IMO not a good choice of equilibrium.
IMO it should be “Machine Psychopharmacology”. Means the same but easier to read.
We’ve mostly phased out questions, because the benefit over just making a post where the title is a question wasn’t worth the additional UI complexity.
I was confused about the “marinade” word usage! I am not quite sure whether I buy it, but it seems like the best explanation I’ve heard so far.
IMO last month has been kind of slow, but April was pretty high, so my guess it’s just random fluctuations (plus things like Inkhaven making a difference and that stopped end of April).
Per the guidelines I wrote in this recent moderation comment, our current policy is that we do not delete general arguments for attacks of this kind, but it is not OK to make direct threats as specific as the one you are making here, and not OK to use the site to coordinate that kind of attack (unless things pretty drastically change about the political climate, as I explain in the linked comment). I redacted your request for people to contact you off-site from both this comment and your bio. Your P.S. is also clearly violating those site norms. I am leaving it up to make the case-law clearer, but we will ban you if you write more things like that.
Again, it is not against our moderation rules to argue for even quite extreme things, but no direct threats, and no coordination for this kind of stuff (barring some pretty truly exceptional circumstances as explained in my comment above).
Do you only feel this way
I am not sure what the question is. I am not presenting particularly specific feelings about politicians here.
Of the two things you list, the former seems better than the latter. The latter seems kind of fake and not how I expect things to actually work? I don’t think politicians can turn themselves into perfect avatars of their constituents’ preferences, so pretending you are seems bad form to me. At the very least there are indexical preferences that make that kind of fundamentally infeasible “I do not want to do this because it is very stressful” does not even quite compute if you try to purely model yourself as the perfect avatar of your constituents’ preferences.
Also, I think in everyday political work, you will just receive appointments that call for different levels of discretion. Being appointed to an advisory board usually calls for using more of your own judgement and models, whereas being appointed to a lawmaker position calls for channeling more of your constituent’s preferences, so I don’t think there is any high-integrity way to act if you can’t model those two as separate.
I think there is a big difference between “Changing your values” and “acting on your constituents’ values”, and while lots of people choose the former in political domains, I think the latter often works similarly well, and is much higher integrity.
I have long been thinking about writing a post on “hat theory”. It’s common and normal for people to act “with my representative hat on” or “with my CEO hat on” or “with my private individual hat on”. I think it’s quite possible, if tricky and a path beset with many traps, to adopt policy priorities within a certain context, but to not adopt them universally, and to be clear about the distinction between what you personally believe and what you advocate on in within a certain societal role.
I think putting both of these in the same bucket is just a mistake. I think you shouldn’t change your values, but nevertheless advocate for things your constituents want, because you are a representative of them.
Not only have we been talking about x-risk consistently for years, but we were also essentially the first ones to systematically talk about it in mainstream media. We were also among the first to push people in the AI safety sphere to be more vocal about it.
Yep, I appreciate that you more recently have been focusing on that. I have made my criticisms of your deepfake campaigns and other related efforts in the past. I am glad you more recently are focusing more on x-risk.
Most of my concerns are not oriented around you not being straightforward about x-risk. You have clearly recently been straightforward about that. High-integrity behavior requires much more than that.
Is it possible that you heard those negative references from people who could have had ulterior motives for opposing the work of ControlAI? Or from people who oppose the concept of advocacy in general due to a pathologically narrow definition of honesty (e.g. all slogans are lies / advertising is equivalent to lying)? In my long experience, those are the only kinds of people who say that PauseAI is dishonest, and ControlAI is even more narrowly focused than that.
Nope, not to my knowledge. Everyone I have talked to is very sympathetic to advocacy and very in favor of calling out extinction risks straightforwardly.
In my long experience, those are the only kinds of people who say that PauseAI is dishonest
What are you talking about? Holly has said dozens of verifiably false things about me, my friends, other people in the ecosystem, other organizations she is collaborating with, and organizations she criticizes. Holly, and by extension PauseAI, is extremely reckless in their regard for the truth.
Honesty is not just measured in how straightforward you are about extinction risks! Being a high-integrity actor requires acting with regards to the truth on many different axes in many different arenas. Holly has I think extremely unambiguously been failing to live up to an acceptable standard of integrity. I think the situation with ControlAI is trickier, and I have fewer public things I can point to, but I really don’t think there is any ambiguity about Holly repeatedly and extensively stretching the truth, and violating various other cooperative norms, in her work. I honestly can’t, at this point, think of anyone who I know independently who has worked with her and would have positive things to say about how that went, and really more than a dozen people who had a really bad time or had major concerns about how she operated.
I have continued to hear pretty consistently the same things since then. It’s hard to get an unbiased sample, but approximately everyone I trust who has had some experience working with them gives really quite strongly negative references about their integrity. The consistency and amount of negative references I got are all pretty outlier-ish for people in the AI safety space (not like unprecedented, but there are maybe like 1-2 other groups where people raise as many warning flags).
Appreciate you replying!
Yes, I also believe this to be the case (though politics changes people, so the Bores who stays in power might not be the Bores who still believes this). It is also one of the things that has left me concerned about his integrity, because in speeches and in public presentation that is not at all what he leads with or communicates, and this strikes me as strategic conflation. If I didn’t believe this I would have less concerns about his integrity!
Like, it seems very obvious to me that the vast majority of Bores’ statements on social media would be considered gross norm violations on LessWrong, because it would be so clear that he would be arguing in bad faith for harms from AI that are not important, but can be helpfully conflated with AI catastrophic and existential risk concerns. This does not strike me as being true of all politician’s public speech, though it does seem true of most. He, for example, strikes me as worse than Obama on the degree to which I am picking up conflation, exaggeration, and some amount of dishonesty in his public engagements. I often got the sense Obama retreated towards broad metaphorical speech when saying what he actually believed would be inconvenient, but did not end up conflating things that matter as directly with things that do not matter for political points (though he sure also did it a bunch).
I have less visibility into his behavior towards other politicians. I was relatively neutral about that, until he proposed the Anti-Insurrectionist-Act which struck me as being particularly partisan and uncooperative towards other politicians, which updated me downwards. But actually reading the bills and making decisions on the technical merits is good and updates me upwards.
I also personally think the strategy of conflating small harms from AI with catastrophic harms is fundamentally short-sighted. My guess is we will soon get many politicians who are very freaked out about AI, it has a decent chance of being a strong bipartisan topic, and the most likely topic on which the parties could come together are the most intense risks. Having Bores as a high-profile politician do a lot of conflation is a big cost that I think only seems worth it because there is so little traction on getting these topics considered seriously right now, but my guess is will seem like a great waste when there will be many many people who will be freaking out about the risks soon and will end up having a harder time orienting due to these risks being conflated.
Whoo boy, yeah, A4550 sure seems like a truly atrocious bill, just having read the summary:
I sure am glad he voted against that. The fact that the bill is so atrocious does make this a bit less of an update, but it’s still significant. Appreciate the data.
Also just following up on this footnote:
I unfortunately think that if you are trying to prevent these two risks, then many interventions are close to diametrically opposed to what you would do if you tried to prevent loss of control risk. Indeed, my sense is these substantially mirror the motivations of the leading AI companies, which has resulted in an extremely intense race towards superintelligence in an attempt to permanently disempower “the bad guys” who would build the bioweapons or establish authoritarianism.[1]
It seems really important for people to know that loss of control doesn’t make the top of his list given that! (Though to be clear, there are lots of regulations one could pass that would help with these two and loss of control, but it indeed requires particular cool-headedness and integrity, which are not the dimensions on which I currently am most excited about Bores).
Though the importance of making trillions of dollars on the way there should not be understated and is probably more responsible for what is actually going on at the labs.