A few weeks ago, I was offered a role as a Staff Research Scientist at Resolution (formerly Sequent) to do agent foundations research. However, they later rescinded the offer due to concerns that having someone with my (publicly expressed) political opinions as an early hire might make it harder for them to hire other researchers. I want to give some brief thoughts on this, starting with the concrete details of what happened.
I talked to Jesse Hoogland, Dan Murfet, and Geoffrey Irving before they gave me the offer. These were fairly informal chats, since they all already knew me. I signed the offer a few days later, then soon afterwards I messaged Jesse about my politics curriculum.[1] Specifically, I suggested that I should run the next iteration of the curriculum before joining Resolution, so that people wouldn’t interpret it as work that I was doing at Resolution.
Jesse noted some controversial aspects of the curriculum,[2] then set up a meeting with me, him, and Dan. I was expecting them to express some concerns, such as asking me not to run it (though I did indirectly mention the possibility that this might put them off from hiring me).[3] Instead, they directly told me that they’d like to rescind the offer.
I had a long conversation with Jesse and Dan, and later a long conversation with Geoffrey. In the latter I learned that, although the decision was prompted by me sending them my curriculum, Geoffrey in particular was more concerned about my tweets. I’ve tweeted some fairly controversial things over the last few years, though the only specific thread Geoffrey cited was thisone (which led to this exchange with Kelsey Piper). It seems like he’s excited about hiring academics who are new to alignment, and worried that those people might not join if they hear about such tweets (though I didn’t get the sense that any specific people had given him such indications). Geoffrey claimed that he would have made the same decision if I’d posted equivalent left-wing takes.
Some high-level thoughts:
It is to Resolution’s credit that they gave me the offer originally despite already having some sense that I was controversial. There are many other organizations that would have ghosted me or rejected me at an earlier stage in the process because of that, and I don’t consider rescinding the offer for political reasons to be significantly worse than not having given it in the first place for political reasons.
I also respect them for taking seriously the idea that there are deep, principled theories of intelligence out there waiting to be discovered—this is a major reason why I wanted to join them in the first place.
I think that Resolution is doing the wrong thing here both in ethical terms, and also in terms of pursuing their proximate goals.
In terms of ethics, I think that commitment to free and open inquiry is an extremely important ethical principle, especially for leaders of research organizations. Decades of erosion of academic freedoms have made it clear how important taking a strong stance on this is. I’m not going to be any less vocal as a result of this, but it’s the kind of thing that makes it fraught for younger (and less independently wealthy) researchers to pursue the truth wherever it leads them.
In terms of their goal of solving alignment: I take these events as a significant update downwards on how good Resolution will be as a place to do open-ended thinking about alignment. My inside view is that the principles of agency are scale-free, so evidence from economics, sociology and politics can significantly improve our work on technical alignment (and vice versa). More generally, the most productive intellectual environments are ones which (like early LessWrong or FHI) are very encouraging of unusual and out-of-the-box thinking. I’ve updated that Resolution cares about this less than I had hoped.
If Resolution mainly wants to hire people to work on already-scoped-out research directions, rather than going for the kind of broader research environment I describe above, that doesn’t seem like a totally unreasonable choice (e.g. ARC seems to be doing this). However, given the amount of money they have and the breadth of hiring they’re attempting, there’s a plausible world where they become the “rallying point” for principled alignment research, while retaining a more restrictive intellectual environment. That seems like a bad outcome.
On a different note, I’ve become increasingly worried that Resolution’s plan to automate alignment research will lead them to produce a lot of capabilities progress (since “automated alignment researcher” and “automated capabilities researcher” are such similar things to aim for). I’d actually written up a doc expressing concern about trying to automate alignment research, and shared it with them during the interview process; however, I didn’t press them on how they planned to avoid this issue.
The kinds of virtue that would make me comfortable with an organization doing a bunch of dual-use research seem pretty strongly anticorrelated with decisions like rescinding my offer. AI alignment will only become more central to the coming power struggles over AI, and being willing to give in to vague concerns about potential hires doesn’t bode well for Resolution’s ability to navigate those struggles in high-integrity ways. (I think Geoffrey would characterize this not as “giving in” but rather as “trying to create a comfortable work environment” or similar. However, the two feel closely related to me. I do view Geoffrey as able to stand up to pressure/local incentives when he decides to, so this is mainly a concern about when and how he’ll decide to.)
In hindsight, given the poor track record of large alignment-branded research organizations, it was a significant mistake for me to accept Resolution’s offer without having thought much more carefully about ways that joining might be bad (such as the points above). There’s a common failure mode whereby rationalists and EAs are averse to thinking about how their cognition is skewed by incentives and power dynamics (especially relative to how clearly these groups think about other things); though I often talk about this issue abstractly, I still failed to account for it in this case.
Given the points above, I can’t in good conscience recommend that people work at Resolution. It’s also worth noting that the weight I place on these points is strongly influenced by my background views on how the alignment community keeps failing to achieve its goals (and often adopts strategies that actively backfire). I’m writing up a much longer and more detailed post on these background views, and intend to post a public version sometime next week.
[Edit: this post originally said “fairly senior role” rather than “Staff Research Scientist”. I changed it because the former might have implied that I would be leading a team or managing people, whereas in fact the role was to be an independent contributor.]
In hindsight I should have talked to them about the curriculum before accepting the offer. My reasoning at the time was that accepting the offer first would send a signal that running the curriculum wasn’t a hard requirement for me. However, I don’t think that message got through.
For context, some of the topics that are covered in the curriculum include the feminization of academia, racial IQ differences, and anti-white racism. Though note that it’s a weird curriculum in a few different directions, rather than being uniformly right-wing—e.g. it has some Lacan-inspired stuff, some hippie-inspired stuff on beliefs as emotional strategies, a bunch of rationalist writing, etc.
The specific message I sent them after learning that were concerned about risks from hiring me:
FWIW I’m pretty familiar with feeling a sense of liability related to what allies or affiliates might say, it was a recurring theme at OpenAI. So I expect I have a reasonable sense of where you’re coming from, and I’m happy to brainstorm ways to mitigate that sense of liability (e.g. not putting Resolution in my twitter bio). More generally, I want to be collaborative on this topic, and won’t hold you to any commitments you guys have made if you want to re-evalaute them now.
I don’t know exactly what risks you’re most concerned about, but it might be useful for me to explain why I personally don’t feel like I’m making a costly tradeoff against making AGI go well by opining on controversial topics online (and in fact believe that it’s meaningfully helpful). I’ll talk about my inside view and an outside view. Also happy to discuss any qualms you might have on a personal level/re office culture tomorrow.
Inside view: for better or worse, a lot of the strategic considerations around the future of AI are actually deeply entangled with the dysfunctions of the wider modern world. For example, academia’s inability to grapple with either AGI or AGI risk seems like a very important dynamic to understand, and that seems quite related to how academia has changed over the last century. As another example, if we want to reason about how more capable entities might disempower less capable ones, probably the best example we have is colonialism—and I think discussions of AI risk would be significantly less vague and abstract if people were less scared of honestly analyzing historical power struggles. (Daniel Kokotajlo’s writings on the conquistadors as precedents for takeover are one big exception, but still miss a lot of the potential insights.)
Outside view: one of the biggest lessons of the last few decades politically, I think, has been that the costs of intellectual taboos are much larger than most people expected—e.g. they meant that the massive populist upheavals of the last decade took almost all intellectuals by surprise. I also think that AI safety is in a much worse position today than it would have been if it’d taken a more principled truth-seeking stance a decade ago (for example, AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA). Indeed, with respect to engaging with the current administration, it seems like standard EA views are more of a liability than any of the things you mentioned by a significant margin, and I expect that having a right-winger at Resolution would be a net gain in political capital. (To be clear, I don’t think that this should be a significant consideration for you, I mention it mainly to highlight that the risks are actually more symmetric than they might seem.)
In general, though, I wouldn’t want to join Resolution if I planned to spend a significant proportion of my time thinking about this stuff. Indeed, a key reason I’m excited about Resolution is because my research has lately been steering towards much more foundational-level questions that I previously expected, and so I intend to be pretty focused on pinning those down.more long
I find some of Richard’s attitudes very distasteful, and if I was running an org and thinking of hiring Richard I would sure be pretty scared about him being seen as a representative of the org or just being off-putting to employees and stakeholders; I think Resolution was absolutely correct to be scared about that. But idk how that should cash out. This situation seems unfortunate. :(
The kinds of virtue that would make me comfortable with an organization doing a bunch of dual-use research seem pretty strongly anticorrelated with decisions like rescinding my offer. AI alignment will only become more central to the coming power struggles over AI, and being willing to give in to vague concerns about potential hires doesn’t bode well for Resolution’s ability to navigate those struggles in high-integrity ways.
I just wanted to say, as a guy sometimes sketched out or confused by Richard’s politics, I agree with this.
I feel particularly worried about it about dual use, but, that’s maybe just a subsection of: If Resolution is trying to be a place doing world class intellectual work, people there will need to think thoughts that go in directions that don’t fit consensus, which are correlated with people finding uncomfortable.
I realize most of what it’s trying to do is in a STEM-y frame that you might theoretically hope is orthogonal to politics. But I expect a lot of people to do their thinking about solving superalignment as part of their thinking about AI macrostrategy, and in general the lines here are kind of blurry.
I also just think… from a sorta normal politics standpoint… The fact that the US is forking into two sides that can’t coexist with each other is part of the problem. If your org culture can’t talk to people like Richard, you are implicitly throwing out like half of America?
There are several different reasons it seems valuable to be proactively setting a culture of “we can talk about a wide variety of ideas here.”
idk there’s a longer better comment here I want to write but don’t really have time atm.
Talking to Richard and hiring Richard are two different things. It’s plausible to me that all these things are true simultaneously:
Richard would do a great job at contributing useful research directions / thoughts / etc.
Richard would be great for the org culture and treat everyone well / respectfully / etc.
Richard’s politics have substantial admixtures of attitudes that are dumb (both incorrect and harmful), and indicative of general attitudes / thought patterns that are reasonably taken as hostile / toxic / etc.
Other good people have low thresholds for that sort of indication, for good reasons, and in fact would be less likely to join and/or fund the org if Richard is there
The expected net effect of hiring Richard is negative, including integrity / upholding principles (it’s not necessarily unprincipled to be like “I think Richard’s ideas are good enough that it’s worth some tail risk, but I get that lots of other good people wouldn’t have that information, and furthermore have good reason to distrust him for political reasons, so I won’t hire him”)
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Yeah the better comment I’d like to have written engages more with the tradeoffs here. I totally agree there are (or at least might be) real tradeoffs here.
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Absolutely I expect there to be potential hires who are turned off. But, I think the current amount that people expect to be able to work at companies without dealing with people who have politics that feel crazy and bad, are waaaaaaaaaaaay out of whack. (I’m not sure if you disagree particularly or are just pointing out the obvious failure modes and issues).
I don’t actually know that I think Resolution should do something different, given the existing founders and culture at Resolution. I think I am advocating or wishing for a high-skill culturebuilding maneuver that is not, like, a free action anyone can just take.
But, I think it is a negative sign abut their capability as an intellectual org, if they aren’t able or willing to do the culture-setting moves I’d be hoping for here.
Mainly I meant to say “you probably should have substantive uncertainty about the extent to which it’s a sign about their capability, because (I assume) you don’t know the sorts of people they’re trying to hire and how those people orient to politics and whether that’s a good orientation or not”.
I would be quite surprised if I were surprised (upon learning more) about what sort of people they were trying to hire, or how those people relate to politics.
I could still totally be overconfident about the overall frame here but, like, I’m already assuming “they are basically correct about the immediate implications of hiring Richard.” Is there a particular thing you’re imagining I might not be imagining?
Richard’s viewpoints seem explicitly incompatible with a majority of the world and a plurality of Americans; if your goal is getting the best researchers overall, a coworker who is actively hostile and antagonistic to approximately 84% of the global population seems like a bad way to cast a wide net.
This is to say nothing about his views on alignment (while I think one could argue that ASI aligned to Richard’s point of view would be unaligned with most of humanity, it’s irrelevant here); he could likely contribute best to the alignment debate from a more scoped organization.
Looking over the [edit: specific] cited tweets and tweet conversation, I don’t get it.
This seems really quite tame? Richard is laying out an abstract taxonomy of views nationhood, and critiquing them. Then he has a cordial exchange with Kelsey, in which she requests and he attempts to pass her ITT (and does a good job according to her). This all seems polite, reasonable, and well within the norms of online intellectual political discourse?
I notice I’m confused.
Are there more tweets that I couldn’t find?
Do I radically misunderstand what reasonable people find offensive?
Is even this level of political analysis verboten in conventional professional circles?
Should I update that resolution has really different standards than I do, or is exceptionally cowardly with regard to possible social attacks?
Do they have a spidey-sense that Richard endorsing the “national right” view is too close to ethnonationalism, which is too close to racism, and that potential employees wouldn’t want to work for Resolution, because Richard is also employed there? (To which I’m like, “Are you kidding me? Even if that’s true, why do you want to work with such people?”)
Are their real concerns about Richard something else, which is inappropriate to cite for some reason, so they’re citing this instead?
It seems like I’m clearly missing something. What’s going on here?
I agree the linked tweet is quite tame. But I looked at the 20 most recent tweets/RTs on Richard’s timeline and they include a RT about the Great Replacement, a RT of the “Department of Deportation” about non-white sex criminals and a RT endorsing race realism.
Indeed, I was expecting stuff more like that when I clicked the link.
Why was this interaction cited as damning, instead of much more overtly edgy stuff? Were they just embarrassed to even bring up that other content, or something, and so pointed to a nothing-burger instead?
1) The cited tweet got more attention (see the interaction with Kelsey Piper)
2) That tweet is by Richard himself whereas the ones I cited are all RTs and I think it’s reasonable to form less strong opinions of people based on what they RT than what they post under their own name.
Do they have a spidey-sense that Richard endorsing the “national right” view is too close to ethnonationalism, which is too close to racism, and that potential employees wouldn’t want to work for Resolution, because Richard is also employed there? (To which I’m like, “Are you kidding me? Even if that’s true, why do you want to work with such people?”)
While I’m not sure I entirely agree with the decision, I don’t think it’s obviously really stupid. Richard’s views, whether you agree with them or not, are very controversial and I think it’s plausible having him in a senior role would indeed discourage others from working for Resolution—especially people who are moving into alignment work and don’t know Richard’s background.
Hmm, this makes me worry that “fairly senior role” might imply that I was going to be running a team or managing people, which was not the case. Have edited accordingly.
It seems to me that the decision not to hire you didn’t cause that many problems.
The right-wing analytical framework seems to have major actual issues, and I expect some of them to be avoidable, e.g. via studying non-Western data and sociologists’ works. For example, the feminization of academia seems to have happened in the USSR, but didn’t cause wokeness in Russia and was partially reversed, and postmodernism’s hazardous forms which caused many problems in the West don’t seem to be as widespread in Russia.
A post-AGI world would cause us to rethink ethics towards the Left far more thoroughly than we expect (e.g. requiring your case for an ethics not so occupied by altruism to explore in more detail what it means to create goodness). For example, if AGIs and robots become aligned and capable of doing whatever work the humans can come up with, then the UBI or another form of taking power away from oligarchs and giving it to regular humans become an absolute necessity. Before the AGIs exist, regular humans have options like bargaining or opening a small business, and the Right have the ability to think that demanding the UBI is a form of entitlement which, if implemented, would be either meaningless or outright dismotivating.
P.S. I wonder if the right-wing framework can outright rule out the ability to align the AIs to the humans’ goals. The world is filled with conspiracy theories claiming that [a hated group like Jews] took over key positions and actively tries to lock in power via mechanisms including an attempt to shift the human culture so that individuals would be unable to coordinate against the hated group.
There’s a common failure mode whereby rationalists and EAs are averse to thinking about how their cognition is skewed by incentives and power dynamics
Man, I do often feel that rationalists, and especially EAs are too willing to be credulous based on social signals that aren’t all that epistemically reliable (and I plan to write about this sometime). But it also seems like rationalists and EAs do better at this than almost any similarly sized subculture?
Most people just uncritically adopt the views of their social peers, without almost any reflection!
The only similarly-sized group that I know of that might be doing better is the set of active political operators in DC? They’re definitely aware of the influence of power. But even so, I think most of them basically uncritically adopt parts of the party line?
I think most onlookers would interpret Richard as saying “Rationalists and EAs are unusually bad at this”. Insofar as Richard thinks that, I want to publicly dispute it. Insofar as he doesn’t, and this is a miscommunication, I want to correct the record.
That rationalists are better than average does not exempt us from the problems caused by our not being remotely good enough, which is why I plan to write about it.
Well, in theory it could for example suggest, directionally, whether it’s more a “my rationalist tools are breaking in my hands” situation vs. a “I have failed to live up to my rationalist tools” sort of situation.
In hindsight, given the poor track record of large alignment-branded research organizations, it was a significant mistake for me to accept Resolution’s offer without having thought much more carefully about ways that joining might be bad (such as the points above). There’s a common failure mode whereby rationalists and EAs are averse to thinking about how their cognition is skewed by incentives and power dynamics; though I often talk about this issue abstractly, I still failed to account for it in this case.
On a different note, I’ve become increasingly worried that Resolution’s plan to automate alignment research will lead them to produce a lot of capabilities progress (since “automated alignment researcher” and “automated capabilities researcher” are such similar things to aim for).
You’ve been in favor of interpretability research in the past, wouldn’t that similarly increase capabilities?
A few weeks ago, I was offered a role as a Staff Research Scientist at Resolution (formerly Sequent) to do agent foundations research. However, they later rescinded the offer due to concerns that having someone with my (publicly expressed) political opinions as an early hire might make it harder for them to hire other researchers. I want to give some brief thoughts on this, starting with the concrete details of what happened.
I talked to Jesse Hoogland, Dan Murfet, and Geoffrey Irving before they gave me the offer. These were fairly informal chats, since they all already knew me. I signed the offer a few days later, then soon afterwards I messaged Jesse about my politics curriculum.[1] Specifically, I suggested that I should run the next iteration of the curriculum before joining Resolution, so that people wouldn’t interpret it as work that I was doing at Resolution.
Jesse noted some controversial aspects of the curriculum,[2] then set up a meeting with me, him, and Dan. I was expecting them to express some concerns, such as asking me not to run it (though I did indirectly mention the possibility that this might put them off from hiring me).[3] Instead, they directly told me that they’d like to rescind the offer.
I had a long conversation with Jesse and Dan, and later a long conversation with Geoffrey. In the latter I learned that, although the decision was prompted by me sending them my curriculum, Geoffrey in particular was more concerned about my tweets. I’ve tweeted some fairly controversial things over the last few years, though the only specific thread Geoffrey cited was this one (which led to this exchange with Kelsey Piper). It seems like he’s excited about hiring academics who are new to alignment, and worried that those people might not join if they hear about such tweets (though I didn’t get the sense that any specific people had given him such indications). Geoffrey claimed that he would have made the same decision if I’d posted equivalent left-wing takes.
Some high-level thoughts:
It is to Resolution’s credit that they gave me the offer originally despite already having some sense that I was controversial. There are many other organizations that would have ghosted me or rejected me at an earlier stage in the process because of that, and I don’t consider rescinding the offer for political reasons to be significantly worse than not having given it in the first place for political reasons.
I also respect them for taking seriously the idea that there are deep, principled theories of intelligence out there waiting to be discovered—this is a major reason why I wanted to join them in the first place.
I think that Resolution is doing the wrong thing here both in ethical terms, and also in terms of pursuing their proximate goals.
In terms of ethics, I think that commitment to free and open inquiry is an extremely important ethical principle, especially for leaders of research organizations. Decades of erosion of academic freedoms have made it clear how important taking a strong stance on this is. I’m not going to be any less vocal as a result of this, but it’s the kind of thing that makes it fraught for younger (and less independently wealthy) researchers to pursue the truth wherever it leads them.
In terms of their goal of solving alignment: I take these events as a significant update downwards on how good Resolution will be as a place to do open-ended thinking about alignment. My inside view is that the principles of agency are scale-free, so evidence from economics, sociology and politics can significantly improve our work on technical alignment (and vice versa). More generally, the most productive intellectual environments are ones which (like early LessWrong or FHI) are very encouraging of unusual and out-of-the-box thinking. I’ve updated that Resolution cares about this less than I had hoped.
If Resolution mainly wants to hire people to work on already-scoped-out research directions, rather than going for the kind of broader research environment I describe above, that doesn’t seem like a totally unreasonable choice (e.g. ARC seems to be doing this). However, given the amount of money they have and the breadth of hiring they’re attempting, there’s a plausible world where they become the “rallying point” for principled alignment research, while retaining a more restrictive intellectual environment. That seems like a bad outcome.
On a different note, I’ve become increasingly worried that Resolution’s plan to automate alignment research will lead them to produce a lot of capabilities progress (since “automated alignment researcher” and “automated capabilities researcher” are such similar things to aim for). I’d actually written up a doc expressing concern about trying to automate alignment research, and shared it with them during the interview process; however, I didn’t press them on how they planned to avoid this issue.
The kinds of virtue that would make me comfortable with an organization doing a bunch of dual-use research seem pretty strongly anticorrelated with decisions like rescinding my offer. AI alignment will only become more central to the coming power struggles over AI, and being willing to give in to vague concerns about potential hires doesn’t bode well for Resolution’s ability to navigate those struggles in high-integrity ways. (I think Geoffrey would characterize this not as “giving in” but rather as “trying to create a comfortable work environment” or similar. However, the two feel closely related to me. I do view Geoffrey as able to stand up to pressure/local incentives when he decides to, so this is mainly a concern about when and how he’ll decide to.)
In hindsight, given the poor track record of large alignment-branded research organizations, it was a significant mistake for me to accept Resolution’s offer without having thought much more carefully about ways that joining might be bad (such as the points above). There’s a common failure mode whereby rationalists and EAs are averse to thinking about how their cognition is skewed by incentives and power dynamics (especially relative to how clearly these groups think about other things); though I often talk about this issue abstractly, I still failed to account for it in this case.
Given the points above, I can’t in good conscience recommend that people work at Resolution. It’s also worth noting that the weight I place on these points is strongly influenced by my background views on how the alignment community keeps failing to achieve its goals (and often adopts strategies that actively backfire). I’m writing up a much longer and more detailed post on these background views, and intend to post a public version sometime next week.
[Edit: this post originally said “fairly senior role” rather than “Staff Research Scientist”. I changed it because the former might have implied that I would be leading a team or managing people, whereas in fact the role was to be an independent contributor.]
In hindsight I should have talked to them about the curriculum before accepting the offer. My reasoning at the time was that accepting the offer first would send a signal that running the curriculum wasn’t a hard requirement for me. However, I don’t think that message got through.
For context, some of the topics that are covered in the curriculum include the feminization of academia, racial IQ differences, and anti-white racism. Though note that it’s a weird curriculum in a few different directions, rather than being uniformly right-wing—e.g. it has some Lacan-inspired stuff, some hippie-inspired stuff on beliefs as emotional strategies, a bunch of rationalist writing, etc.
The specific message I sent them after learning that were concerned about risks from hiring me:
FWIW I’m pretty familiar with feeling a sense of liability related to what allies or affiliates might say, it was a recurring theme at OpenAI. So I expect I have a reasonable sense of where you’re coming from, and I’m happy to brainstorm ways to mitigate that sense of liability (e.g. not putting Resolution in my twitter bio). More generally, I want to be collaborative on this topic, and won’t hold you to any commitments you guys have made if you want to re-evalaute them now.
I don’t know exactly what risks you’re most concerned about, but it might be useful for me to explain why I personally don’t feel like I’m making a costly tradeoff against making AGI go well by opining on controversial topics online (and in fact believe that it’s meaningfully helpful). I’ll talk about my inside view and an outside view. Also happy to discuss any qualms you might have on a personal level/re office culture tomorrow.
Inside view: for better or worse, a lot of the strategic considerations around the future of AI are actually deeply entangled with the dysfunctions of the wider modern world. For example, academia’s inability to grapple with either AGI or AGI risk seems like a very important dynamic to understand, and that seems quite related to how academia has changed over the last century. As another example, if we want to reason about how more capable entities might disempower less capable ones, probably the best example we have is colonialism—and I think discussions of AI risk would be significantly less vague and abstract if people were less scared of honestly analyzing historical power struggles. (Daniel Kokotajlo’s writings on the conquistadors as precedents for takeover are one big exception, but still miss a lot of the potential insights.)
Outside view: one of the biggest lessons of the last few decades politically, I think, has been that the costs of intellectual taboos are much larger than most people expected—e.g. they meant that the massive populist upheavals of the last decade took almost all intellectuals by surprise. I also think that AI safety is in a much worse position today than it would have been if it’d taken a more principled truth-seeking stance a decade ago (for example, AI governance would have been less likely to throw in with the Democrats in a way that alienated MAGA). Indeed, with respect to engaging with the current administration, it seems like standard EA views are more of a liability than any of the things you mentioned by a significant margin, and I expect that having a right-winger at Resolution would be a net gain in political capital. (To be clear, I don’t think that this should be a significant consideration for you, I mention it mainly to highlight that the risks are actually more symmetric than they might seem.)
In general, though, I wouldn’t want to join Resolution if I planned to spend a significant proportion of my time thinking about this stuff. Indeed, a key reason I’m excited about Resolution is because my research has lately been steering towards much more foundational-level questions that I previously expected, and so I intend to be pretty focused on pinning those down.more long
I find some of Richard’s attitudes very distasteful, and if I was running an org and thinking of hiring Richard I would sure be pretty scared about him being seen as a representative of the org or just being off-putting to employees and stakeholders; I think Resolution was absolutely correct to be scared about that. But idk how that should cash out. This situation seems unfortunate. :(
I just wanted to say, as a guy sometimes sketched out or confused by Richard’s politics, I agree with this.
I feel particularly worried about it about dual use, but, that’s maybe just a subsection of: If Resolution is trying to be a place doing world class intellectual work, people there will need to think thoughts that go in directions that don’t fit consensus, which are correlated with people finding uncomfortable.
I realize most of what it’s trying to do is in a STEM-y frame that you might theoretically hope is orthogonal to politics. But I expect a lot of people to do their thinking about solving superalignment as part of their thinking about AI macrostrategy, and in general the lines here are kind of blurry.
I also just think… from a sorta normal politics standpoint… The fact that the US is forking into two sides that can’t coexist with each other is part of the problem. If your org culture can’t talk to people like Richard, you are implicitly throwing out like half of America?
There are several different reasons it seems valuable to be proactively setting a culture of “we can talk about a wide variety of ideas here.”
idk there’s a longer better comment here I want to write but don’t really have time atm.
Talking to Richard and hiring Richard are two different things. It’s plausible to me that all these things are true simultaneously:
Richard would do a great job at contributing useful research directions / thoughts / etc.
Richard would be great for the org culture and treat everyone well / respectfully / etc.
Richard’s politics have substantial admixtures of attitudes that are dumb (both incorrect and harmful), and indicative of general attitudes / thought patterns that are reasonably taken as hostile / toxic / etc.
Other good people have low thresholds for that sort of indication, for good reasons, and in fact would be less likely to join and/or fund the org if Richard is there
The expected net effect of hiring Richard is negative, including integrity / upholding principles (it’s not necessarily unprincipled to be like “I think Richard’s ideas are good enough that it’s worth some tail risk, but I get that lots of other good people wouldn’t have that information, and furthermore have good reason to distrust him for political reasons, so I won’t hire him”)
I would not guess that all of these are true, but I wouldn’t be so confident without, for example, having talked to several potential hires for the org.
Yeah the better comment I’d like to have written engages more with the tradeoffs here. I totally agree there are (or at least might be) real tradeoffs here.
Absolutely I expect there to be potential hires who are turned off. But, I think the current amount that people expect to be able to work at companies without dealing with people who have politics that feel crazy and bad, are waaaaaaaaaaaay out of whack. (I’m not sure if you disagree particularly or are just pointing out the obvious failure modes and issues).
I don’t actually know that I think Resolution should do something different, given the existing founders and culture at Resolution. I think I am advocating or wishing for a high-skill culturebuilding maneuver that is not, like, a free action anyone can just take.
But, I think it is a negative sign abut their capability as an intellectual org, if they aren’t able or willing to do the culture-setting moves I’d be hoping for here.
Mainly I meant to say “you probably should have substantive uncertainty about the extent to which it’s a sign about their capability, because (I assume) you don’t know the sorts of people they’re trying to hire and how those people orient to politics and whether that’s a good orientation or not”.
I would be quite surprised if I were surprised (upon learning more) about what sort of people they were trying to hire, or how those people relate to politics.
I could still totally be overconfident about the overall frame here but, like, I’m already assuming “they are basically correct about the immediate implications of hiring Richard.” Is there a particular thing you’re imagining I might not be imagining?
Richard’s viewpoints seem explicitly incompatible with a majority of the world and a plurality of Americans; if your goal is getting the best researchers overall, a coworker who is actively hostile and antagonistic to approximately 84% of the global population seems like a bad way to cast a wide net.
This is to say nothing about his views on alignment (while I think one could argue that ASI aligned to Richard’s point of view would be unaligned with most of humanity, it’s irrelevant here); he could likely contribute best to the alignment debate from a more scoped organization.
Looking over the [edit: specific] cited tweets and tweet conversation, I don’t get it.
This seems really quite tame? Richard is laying out an abstract taxonomy of views nationhood, and critiquing them. Then he has a cordial exchange with Kelsey, in which she requests and he attempts to pass her ITT (and does a good job according to her). This all seems polite, reasonable, and well within the norms of online intellectual political discourse?
I notice I’m confused.
Are there more tweets that I couldn’t find?
Do I radically misunderstand what reasonable people find offensive?
Is even this level of political analysis verboten in conventional professional circles?
Should I update that resolution has really different standards than I do, or is exceptionally cowardly with regard to possible social attacks?
Do they have a spidey-sense that Richard endorsing the “national right” view is too close to ethnonationalism, which is too close to racism, and that potential employees wouldn’t want to work for Resolution, because Richard is also employed there? (To which I’m like, “Are you kidding me? Even if that’s true, why do you want to work with such people?”)
Are their real concerns about Richard something else, which is inappropriate to cite for some reason, so they’re citing this instead?
It seems like I’m clearly missing something. What’s going on here?
I agree the linked tweet is quite tame. But I looked at the 20 most recent tweets/RTs on Richard’s timeline and they include a RT about the Great Replacement, a RT of the “Department of Deportation” about non-white sex criminals and a RT endorsing race realism.
Indeed, I was expecting stuff more like that when I clicked the link.
Why was this interaction cited as damning, instead of much more overtly edgy stuff? Were they just embarrassed to even bring up that other content, or something, and so pointed to a nothing-burger instead?
My guess:
1) The cited tweet got more attention (see the interaction with Kelsey Piper)
2) That tweet is by Richard himself whereas the ones I cited are all RTs and I think it’s reasonable to form less strong opinions of people based on what they RT than what they post under their own name.
Yes.
Disappointed to hear about this. Sounds like a really stupid decision from Resolution.
While I’m not sure I entirely agree with the decision, I don’t think it’s obviously really stupid. Richard’s views, whether you agree with them or not, are very controversial and I think it’s plausible having him in a senior role would indeed discourage others from working for Resolution—especially people who are moving into alignment work and don’t know Richard’s background.
Hmm, this makes me worry that “fairly senior role” might imply that I was going to be running a team or managing people, which was not the case. Have edited accordingly.
This seems very disappointing
It seems to me that the decision not to hire you didn’t cause that many problems.
The right-wing analytical framework seems to have major actual issues, and I expect some of them to be avoidable, e.g. via studying non-Western data and sociologists’ works. For example, the feminization of academia seems to have happened in the USSR, but didn’t cause wokeness in Russia and was partially reversed, and postmodernism’s hazardous forms which caused many problems in the West don’t seem to be as widespread in Russia.
A post-AGI world would cause us to rethink ethics towards the Left far more thoroughly than we expect (e.g. requiring your case for an ethics not so occupied by altruism to explore in more detail what it means to create goodness). For example, if AGIs and robots become aligned and capable of doing whatever work the humans can come up with, then the UBI or another form of taking power away from oligarchs and giving it to regular humans become an absolute necessity. Before the AGIs exist, regular humans have options like bargaining or opening a small business, and the Right have the ability to think that demanding the UBI is a form of entitlement which, if implemented, would be either meaningless or outright dismotivating.
P.S. I wonder if the right-wing framework can outright rule out the ability to align the AIs to the humans’ goals. The world is filled with conspiracy theories claiming that [a hated group like Jews] took over key positions and actively tries to lock in power via mechanisms including an attempt to shift the human culture so that individuals would be unable to coordinate against the hated group.
Man, I do often feel that rationalists, and especially EAs are too willing to be credulous based on social signals that aren’t all that epistemically reliable (and I plan to write about this sometime). But it also seems like rationalists and EAs do better at this than almost any similarly sized subculture?
Most people just uncritically adopt the views of their social peers, without almost any reflection!
The only similarly-sized group that I know of that might be doing better is the set of active political operators in DC? They’re definitely aware of the influence of power. But even so, I think most of them basically uncritically adopt parts of the party line?
What is the relevance of this? It is written of the eighth virtue of humility that “it is useless to be superior [...] There is no guarantee that adequacy is possible given your hardest effort; therefore spare no thought for whether others are doing worse.”
I think most onlookers would interpret Richard as saying “Rationalists and EAs are unusually bad at this”. Insofar as Richard thinks that, I want to publicly dispute it. Insofar as he doesn’t, and this is a miscommunication, I want to correct the record.
That rationalists are better than average does not exempt us from the problems caused by our not being remotely good enough, which is why I plan to write about it.
Well, in theory it could for example suggest, directionally, whether it’s more a “my rationalist tools are breaking in my hands” situation vs. a “I have failed to live up to my rationalist tools” sort of situation.
Good point, I’ve added a parenthetical: “(especially relative to how clearly these groups think about other things)”
I’d be very interested in a 5-whys on this.
You’ve been in favor of interpretability research in the past, wouldn’t that similarly increase capabilities?
Very disappointing, sad to hear this!
I think this feels similar to the thread with LongView Philanthropy (linked below). I don’t think this reflects positively on any involved party.
https://www.lesswrong.com/posts/FuGfR3jL3sw6r8kB4/richard-ngo-s-shortform?commentId=RHDvund5npvwWgyAz