The kinds of virtue that would make me comfortable with an organization doing a bunch of dual-use research seem pretty strongly anticorrelated with decisions like rescinding my offer. AI alignment will only become more central to the coming power struggles over AI, and being willing to give in to vague concerns about potential hires doesn’t bode well for Resolution’s ability to navigate those struggles in high-integrity ways.
I just wanted to say, as a guy sometimes sketched out or confused by Richard’s politics, I agree with this.
I feel particularly worried about it about dual use, but, that’s maybe just a subsection of: If Resolution is trying to be a place doing world class intellectual work, people there will need to think thoughts that go in directions that don’t fit consensus, which are correlated with people finding uncomfortable.
I realize most of what it’s trying to do is in a STEM-y frame that you might theoretically hope is orthogonal to politics. But I expect a lot of people to do their thinking about solving superalignment as part of their thinking about AI macrostrategy, and in general the lines here are kind of blurry.
I also just think… from a sorta normal politics standpoint… The fact that the US is forking into two sides that can’t coexist with each other is part of the problem. If your org culture can’t talk to people like Richard, you are implicitly throwing out like half of America?
There are several different reasons it seems valuable to be proactively setting a culture of “we can talk about a wide variety of ideas here.”
idk there’s a longer better comment here I want to write but don’t really have time atm.
Talking to Richard and hiring Richard are two different things. It’s plausible to me that all these things are true simultaneously:
Richard would do a great job at contributing useful research directions / thoughts / etc.
Richard would be great for the org culture and treat everyone well / respectfully / etc.
Richard’s politics have substantial admixtures of attitudes that are dumb (both incorrect and harmful), and indicative of general attitudes / thought patterns that are reasonably taken as hostile / toxic / etc.
Other good people have low thresholds for that sort of indication, for good reasons, and in fact would be less likely to join and/or fund the org if Richard is there
The expected net effect of hiring Richard is negative, including integrity / upholding principles (it’s not necessarily unprincipled to be like “I think Richard’s ideas are good enough that it’s worth some tail risk, but I get that lots of other good people wouldn’t have that information, and furthermore have good reason to distrust him for political reasons, so I won’t hire him”)
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Yeah the better comment I’d like to have written engages more with the tradeoffs here. I totally agree there are (or at least might be) real tradeoffs here.
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Absolutely I expect there to be potential hires who are turned off. But, I think the current amount that people expect to be able to work at companies without dealing with people who have politics that feel crazy and bad, are waaaaaaaaaaaay out of whack. (I’m not sure if you disagree particularly or are just pointing out the obvious failure modes and issues).
I don’t actually know that I think Resolution should do something different, given the existing founders and culture at Resolution. I think I am advocating or wishing for a high-skill culturebuilding maneuver that is not, like, a free action anyone can just take.
But, I think it is a negative sign abut their capability as an intellectual org, if they aren’t able or willing to do the culture-setting moves I’d be hoping for here.
Mainly I meant to say “you probably should have substantive uncertainty about the extent to which it’s a sign about their capability, because (I assume) you don’t know the sorts of people they’re trying to hire and how those people orient to politics and whether that’s a good orientation or not”.
I would be quite surprised if I were surprised (upon learning more) about what sort of people they were trying to hire, or how those people relate to politics.
I could still totally be overconfident about the overall frame here but, like, I’m already assuming “they are basically correct about the immediate implications of hiring Richard.” Is there a particular thing you’re imagining I might not be imagining?
Richard’s viewpoints seem explicitly incompatible with a majority of the world and a plurality of Americans; if your goal is getting the best researchers overall, a coworker who is actively hostile and antagonistic to approximately 84% of the global population seems like a bad way to cast a wide net.
This is to say nothing about his views on alignment (while I think one could argue that ASI aligned to Richard’s point of view would be unaligned with most of humanity, it’s irrelevant here); he could likely contribute best to the alignment debate from a more scoped organization.
The org is trying to hire Agent Foundations researchers. Suppose that becoming such a researcher requires being educated, but the education system usually infects the candidates with leftism in a manner which doesn’t affect their ability to make advancements. Then Ngo would end up hostile to 84% of potential candidates and not to 84% of the population.
As far as I understand Ngo, he worries that leftism either does affect the ability to make advancements or is a demonstration that the researchers are stumped by lack of diversity. For example, if the researcher tried to extrapolate human mechanisms of value emergence onto the AIs while believing in the wrong mechanisms, then the researcher would produce slop. Or, taking an example from my comment, if a sociologist tried to explain wokeness with feminization of academia, then the true explanation would have to account for other factors which the West, unlike Russia, experienced.
P.S. Ngo also claims that “the weight I place on these points is strongly influenced by my background views on how the alignment community keeps failing to achieve its goals (and often adopts strategies that actively backfire). I’m writing up a much longer and more detailed post on these background views, and intend to post a public version sometime next week.” Alas, the three examples which he cites aren’t THAT clear to me: academia’s inability to grapple with either AGI or AGI risk, the costs of intellectual taboos which are much larger than most people expected and the idea that “AI safety is in a much worse position today than it would have been if it’d taken a more principled truth-seeking stance a decade ago (for example, AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA).” The third point is complex because IMHO AI governance does require international coordination to avoid the AI-2027-like race. The first point has a benign explanation.
I think “actively hostile and antagonistic” is unfair, but even if it were true, majority and might do not make right, in politics, rationality, AI alignment, or anything else. There are plenty of important issues and beliefs of all kinds for which most people are deeply mistaken about. Coming to terms with this fact can sometimes make people cynical or despairing, or make them come across as hostile / alienating, but I don’t think that’s actually true of Richard or most other folks around here.
(OTOH, not acknowledging that most people are wrong about some pretty fundamental things and then grappling honestly with the implications of that is usually incompatible with truth-seeking, and can come across as patronizing to the people and groups one claims to empathize with.)
My guess is Geoffrey do agree that it is important for AI safety orgs like Resolution to be more diverse intellectually, and it is in fact the reason why he made that decision. Richard himself notes that “[Geofrey]’s excited about hiring academics who are new to alignment, and worried that those people might not join if they hear about such tweets”.
Neoreactionaries (not claiming Ngo would necessarily identify as NRx specifically, but he is broadly in that direction, and it is dishonest to claim him as a reasonable stand-in for your average US Republican voter) have been closely associated with rationalist-adjacent spaces since inception, so any ideas they might have had have long been ingested in the discourse. Wanting less of NRx-adjacent stuff around is definitely one of the things you want to do if you want to move away from Silicon Valley monoculture and be more welcoming to your median non-rat/EA academic.
I do not particularly think affirmative action for conservatives make sense in a technical safety organization, but if you want conservative strands which have been hitherto underrepresented in rationalist discourse, you would need to hire religious conservatives and Bannon/Allen populist types, not people who are really into race and IQ. (Libertarians and national security hawks are also pretty well-covered already.)
I just wanted to say, as a guy sometimes sketched out or confused by Richard’s politics, I agree with this.
I feel particularly worried about it about dual use, but, that’s maybe just a subsection of: If Resolution is trying to be a place doing world class intellectual work, people there will need to think thoughts that go in directions that don’t fit consensus, which are correlated with people finding uncomfortable.
I realize most of what it’s trying to do is in a STEM-y frame that you might theoretically hope is orthogonal to politics. But I expect a lot of people to do their thinking about solving superalignment as part of their thinking about AI macrostrategy, and in general the lines here are kind of blurry.
I also just think… from a sorta normal politics standpoint… The fact that the US is forking into two sides that can’t coexist with each other is part of the problem. If your org culture can’t talk to people like Richard, you are implicitly throwing out like half of America?
There are several different reasons it seems valuable to be proactively setting a culture of “we can talk about a wide variety of ideas here.”
idk there’s a longer better comment here I want to write but don’t really have time atm.
Talking to Richard and hiring Richard are two different things. It’s plausible to me that all these things are true simultaneously:
Richard would do a great job at contributing useful research directions / thoughts / etc.
Richard would be great for the org culture and treat everyone well / respectfully / etc.
Richard’s politics have substantial admixtures of attitudes that are dumb (both incorrect and harmful), and indicative of general attitudes / thought patterns that are reasonably taken as hostile / toxic / etc.
Other good people have low thresholds for that sort of indication, for good reasons, and in fact would be less likely to join and/or fund the org if Richard is there
The expected net effect of hiring Richard is negative, including integrity / upholding principles (it’s not necessarily unprincipled to be like “I think Richard’s ideas are good enough that it’s worth some tail risk, but I get that lots of other good people wouldn’t have that information, and furthermore have good reason to distrust him for political reasons, so I won’t hire him”)
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Yeah the better comment I’d like to have written engages more with the tradeoffs here. I totally agree there are (or at least might be) real tradeoffs here.
Absolutely I expect there to be potential hires who are turned off. But, I think the current amount that people expect to be able to work at companies without dealing with people who have politics that feel crazy and bad, are waaaaaaaaaaaay out of whack. (I’m not sure if you disagree particularly or are just pointing out the obvious failure modes and issues).
I don’t actually know that I think Resolution should do something different, given the existing founders and culture at Resolution. I think I am advocating or wishing for a high-skill culturebuilding maneuver that is not, like, a free action anyone can just take.
But, I think it is a negative sign abut their capability as an intellectual org, if they aren’t able or willing to do the culture-setting moves I’d be hoping for here.
Mainly I meant to say “you probably should have substantive uncertainty about the extent to which it’s a sign about their capability, because (I assume) you don’t know the sorts of people they’re trying to hire and how those people orient to politics and whether that’s a good orientation or not”.
I would be quite surprised if I were surprised (upon learning more) about what sort of people they were trying to hire, or how those people relate to politics.
I could still totally be overconfident about the overall frame here but, like, I’m already assuming “they are basically correct about the immediate implications of hiring Richard.” Is there a particular thing you’re imagining I might not be imagining?
Richard’s viewpoints seem explicitly incompatible with a majority of the world and a plurality of Americans; if your goal is getting the best researchers overall, a coworker who is actively hostile and antagonistic to approximately 84% of the global population seems like a bad way to cast a wide net.
This is to say nothing about his views on alignment (while I think one could argue that ASI aligned to Richard’s point of view would be unaligned with most of humanity, it’s irrelevant here); he could likely contribute best to the alignment debate from a more scoped organization.
How did you get this number?? From what I know about Richard Ngo’s positions, it seems a very surprising estimate (but maybe I’m missing something).
The org is trying to hire Agent Foundations researchers. Suppose that becoming such a researcher requires being educated, but the education system usually infects the candidates with leftism in a manner which doesn’t affect their ability to make advancements. Then Ngo would end up hostile to 84% of potential candidates and not to 84% of the population.
As far as I understand Ngo, he worries that leftism either does affect the ability to make advancements or is a demonstration that the researchers are stumped by lack of diversity. For example, if the researcher tried to extrapolate human mechanisms of value emergence onto the AIs while believing in the wrong mechanisms, then the researcher would produce slop. Or, taking an example from my comment, if a sociologist tried to explain wokeness with feminization of academia, then the true explanation would have to account for other factors which the West, unlike Russia, experienced.
P.S. Ngo also claims that “the weight I place on these points is strongly influenced by my background views on how the alignment community keeps failing to achieve its goals (and often adopts strategies that actively backfire). I’m writing up a much longer and more detailed post on these background views, and intend to post a public version sometime next week.” Alas, the three examples which he cites aren’t THAT clear to me: academia’s inability to grapple with either AGI or AGI risk, the costs of intellectual taboos which are much larger than most people expected and the idea that “AI safety is in a much worse position today than it would have been if it’d taken a more principled truth-seeking stance a decade ago (for example, AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA).” The third point is complex because IMHO AI governance does require international coordination to avoid the AI-2027-like race. The first point has a benign explanation.
I think “actively hostile and antagonistic” is unfair, but even if it were true, majority and might do not make right, in politics, rationality, AI alignment, or anything else. There are plenty of important issues and beliefs of all kinds for which most people are deeply mistaken about. Coming to terms with this fact can sometimes make people cynical or despairing, or make them come across as hostile / alienating, but I don’t think that’s actually true of Richard or most other folks around here.
(OTOH, not acknowledging that most people are wrong about some pretty fundamental things and then grappling honestly with the implications of that is usually incompatible with truth-seeking, and can come across as patronizing to the people and groups one claims to empathize with.)
My guess is Geoffrey do agree that it is important for AI safety orgs like Resolution to be more diverse intellectually, and it is in fact the reason why he made that decision. Richard himself notes that “[Geofrey]’s excited about hiring academics who are new to alignment, and worried that those people might not join if they hear about such tweets”.
Neoreactionaries (not claiming Ngo would necessarily identify as NRx specifically, but he is broadly in that direction, and it is dishonest to claim him as a reasonable stand-in for your average US Republican voter) have been closely associated with rationalist-adjacent spaces since inception, so any ideas they might have had have long been ingested in the discourse. Wanting less of NRx-adjacent stuff around is definitely one of the things you want to do if you want to move away from Silicon Valley monoculture and be more welcoming to your median non-rat/EA academic.
I do not particularly think affirmative action for conservatives make sense in a technical safety organization, but if you want conservative strands which have been hitherto underrepresented in rationalist discourse, you would need to hire religious conservatives and Bannon/Allen populist types, not people who are really into race and IQ. (Libertarians and national security hawks are also pretty well-covered already.)