Related Discussion from Thomas Kwa’s MIRI Research Experience
One of the subthreads in Thomas Kwa’s MIRI research experience was about his experience working with Nate. In the comments, some other people brought up their own negative experiences. There was a lot of ensuing discussion about it.
Thomas felt this was distracting from the points he was most interested in (e.g. how infohazard policies slow down research, how new researchers can stop flailing around so much, whether deconfusion is a bottleneck to alignment, or the sharp left turn, etc).
I also somewhat regretted curating the post since we normally avoid curating “community politics” posts, and while the post had a lot of timeless content in both the OP and the discussion, it ended up being a major focus of the comments.
So, I’m moving those comments to this escape-valve-post, where the discussion can continue in whatever direction people end up taking while leaving the original post to focus on more timeless topics that are relevant whether or not you’re plugged into particular social scenes.
I perceive some amount of “giving up” on maintaining social incentives in this comment. I think that’s a mistake, especially when the people are in positions of great power and status in this community.
I think the quoted passage advances an attitude which, in general, allows community invasion by malefactors and other bad actors. Social norms are present for a reason. I think it’s reasonable and healthy to expect people to engage in a respectful and professional manner.
If some individual (like Nate) finds these norms costly for some reason, then that shouldn’t mean “banishment” or “conclude they have bad intent” so much as—at the minimum—”they should clearly communicate their non-respectful/-kind alternative communication protocols beforehand, and they should help the other person maintain their boundaries; if not, face the normal consequences for being rude, like ‘proportional loss of social regard’.”[1]
I, personally, have been on the receiving end of (what felt to me like) a Nate-bulldozing, which killed my excitement for engaging with the MIRI-sphere, and also punctured my excitement for doing alignment theory. (Relatedly, Eliezer doing the “state non-obvious takes in an obvious tone, and decline to elaborate” thing which Thomas mentioned earlier.) Nate did later give his condolences for the bulldozing and for not informing me of his communication style before the conversation, which I appreciated.
But from what I’ve heard, a lot of people have had emotionally bad experiences talking with Nate about alignment research.
I think “communicate before interacting” still runs into failures like “many people (including myself) aren’t sufficiently good at maintaining their own boundaries, such that they actually back out if it starts feeling bad.”
Plus, someone might sunk-cost themselves about the conversation and the marginal cost of additional emotional damage.
Alright, I’ll say it.
I did office operations at MIRI from Sep 2017 to June 2018 as a contractor and it finally feels right to share. All views herein are my own and not meant to represent anyone else. I intended to write a few paragraphs here but ended up with several pages.
Okay, so...my gut wants me to shout, “He’s not simply overly blunt in math arguments! He’s mean and scary[1] toward ops workers! Doesn’t anyone notice this?! I feel like I’m taking crazy pills!” That’s my gut reaction. My reasoned words below will be longer and more nuanced[2].
I wish some people at MIRI had explicitly told me up front something like, “Hey, if you mess up a lunch order, you may want to avoid Nate until the next day. He is a very high-performing researcher, and you should not expect typical levels of patience or anger management from him. Also, if you try to stand up for yourself, he may simply cut you off and storm out of the room. Such is the price of having world-savers…do you have thick enough skin to work here?” And I would have said, “Ah, I appreciate the candor and respect. Seems like you guys are making a reasonable tradeoff—after all Newton was notoriously prickly too. But I’m probably a bad fit for the role. Thanks for your consideration, and I look forward to seeing y’all at the next house party :]”
I spent over a week training my replacement at MIRI, and I feel pretty bad that I didn’t give him that exact warning. I think I said something like, “yeah, by the way, make sure to include such-and-such when you place the lunch order, or else Nate will get real mad at ya, haha. Anyway, moving on…”
(I really feel like I failed my fellow opsman here, so I called him a few hours ago and apologized. He doesn’t feel like I failed him, but he understands why I feel that way, and he appreciated the apology. Also, for the record, some of my fellow ops team said they liked the work I did and were sad to see me go, and I think they were sincere. MIRI was ready to take me on as a full-time employee but I backed out. I did like them too, and wished I could have been around them more and around Nate less, but that wasn’t what the org needed.)
I’ll try to sharpen up my point: I believe that different people are subject to different rules, regardless of official messaging. I claim that while I was at MIRI, Nate was not subject to the same behavioral regulation norms that almost everyone else in the community[3] is. He got away with angry outbursts and defections against norms of cooperative communication, over small stakes such as running out of sourdough bread too often. I think lots of people (especially new people) are not properly warned about this situation, which allows Nate to tilt the usually shared responsibility of self-control onto others in ways that they would not have agreed to if they had been better informed. Of all the things that have increased my cynicism toward the EA ecosystem over the years, none has disturbed me quite as much as the ongoing euphemisms and narrative spin around Nate’s behavior.
I feel like I’ve said what needs to be said, but for the sake of thoroughness, and because people will reasonably want them anyway, here are my object-level complaints. (I feel a little more exposed writing this part than the rest of it. You know how these things tend to be.) :
Once I was delayed a day or two in one of my biweekly grocery runs for the kitchen. Nate had already been grumpy about his favorite bread running out too often, and when a couple other things dried up that week, he...flipped out. He didn’t exactly yell at me and my fellow ops coworker, according to my imaginary decibelmeter, but he was indisputably hostile and aggressive, and obviously uninterested in 2-way communication. I did not have control over the food budget, and if I did, I would have been happy to just buy more food so we would run out less. (This was in the days of much less funding.) I speculate that this particular episode was splash damage from Nate’s conflict with the higher-up ops staff, but I don’t know, and that doesn’t really matter from my perspective.
My manager asked if it would be fine for me to add bike tire inflation to my task list. I was happy to. Nate later complained a couple times that I was leaving his tires underinflated. I asked him if there was some special trick for using his bike pump, because it felt harder to use than the ones I had owned, and I couldn’t detect any user error on my part. He said no, just do it in the straightforward way every so often. I was confused, but further questions felt unwelcome and it seemed possible that I really did just lose track or something. Anyway, one day Nate got a flat tire out on the road, which is dangerous, and he was understandably upset. He complained to my manager who informed me. I saw Nate in the office kitchen later that day (a Saturday) and thought it was an appropriate time to bring up again that I was having trouble with our available pump. I didn’t know how to–“Learn!” he snapped and then stormed out of the room. I considered this moment a major…update about the organization culture, communication standards, and my value in the org. After this incident, the ops team authorized me to purchase a reliable pressure gauge (I don’t remember if I specifically asked them if I could before that, but overall I felt like I couldn’t simply buy things to make my tasks easier without a very clear and legible need). Ever after, I have rolled my eyes at the suggestion that Nate has anything approaching normal levels of self-control or self-understanding. I am containing the urge to use much stronger language about this.
He got really angry at me when the rest of the office outvoted him on the choice of lunch catering. He had veto power on restaurant selection, and that was fine with me, but the anger was apparently not helping him remember to use it when it mattered.
There were other incidents. It was a pattern. I didn’t feel like there was anything I could do except quit, which I did, and which I think he wanted.
Here are a few miscellaneous notes:
With Eliezer, my experience has been the opposite. When I showed up in Berkeley, people who knew Eliezer tripped over themselves to tell me how arrogant and difficult they found him. I’ve talked to him for 5-10 minutes on 5-10 occasions, and every single time he was somewhere between neutral and friendly. I guess I should admit that this was never before lunch, and he was hardly ever around the office anyway, so the comparison is weak.
I once did operations at an early-stage tech startup. On a few occasions the cofounders yelled at us. It honestly didn’t feel too bad at the time, and I don’t hold it against them. The stakes felt real—the company was not yet profitable and we needed to change that ASAP or the company would stop existing. At no point were my expectations painfully violated, and at no point did I feel like my default communication norms were choked for no reason.
I actually had heard of or observed a few of Nate’s scary moments, going back to 2015. I feel ashamed that I was so slow to update on them. My best guess, based on fallible memory, is that by mid-2017 my overall impression of him was confused and uncertain: he had so much social approval, he had written an inspiring and uplifting blog, hell, he had done some quite nice things for me personally!…but there was also this disturbing thread of bad behavior. I didn’t interact with him much and I didn’t work for MIRI yet, so the cognitive dissonance didn’t feel urgent and I simply didn’t think about it much. I count this as a strike against my own judgment: however adaptive my cognition might be at any given time, I now question my ability to notice and call out bad behavior until it personally impacts me.
I’m highly uncertain about how Nate perceives his relationships to operations staff. Will all of my claims simply hit his ear as lies? Understandable confabulations of a disgruntled former contractor? Just a bunch of wild misunderstandings? I honestly don’t know, and I’ll withhold my more detailed speculations.
I could say a lot more, but this is already way too long.
Until now I have mostly kept all these things to myself, for the following reasons:
I didn’t want the trouble. Given that Nate was benefiting what I saw as a flagrantly false public image of self-control, I worried about what I would be getting myself into. It seemed like I would be initiating a conflict with someone who at best was not subject to the same rules as everyone else.
I didn’t think people would listen. After all, I didn’t listen until it directly impacted me.
Didn’t want to interfere with saving the world. But in light of Kwa’s post I now believe this was wrong.
I dunno when I’ll next check this comment thread. Might be tomorrow. Might be never. I wrote all of the above because I honestly believe it, and it finally feels worth the potential trouble to say.
[1]I considered the word “coercive”. But I exist in a free-ish labor market, no one coerced me into working for MIRI in particular. I left due to multiple reasons, including that I didn’t like how Nate conducted himself. That’s the labor market functioning properly. This is really important to me. There is a sense in which people with low market power are “forced” to work at lower compensation/conditions than people with high market power. But that “force” is being applied by every single potential employer who does not offer them a better deal, and I believe that any moral responsibility for a worker’s dissatisfaction is commensurately diffuse.
[2]I have some concern that discussions of Nate’s behavior often have a unusually high level of nuance and circumspection, in a way that makes me suspect motivated continuation or similar.
[3]Even given his rank and station.
Thanks for saying so!
My intent was not to make you feel bad. I apologize for that, and am saddened by it.
(I’d love to say “and I’ve identified the source of the problem and successfully addressed it”, but I don’t think I have! I do think I’ve gotten a little better at avoiding this sort of thing with time and practice. I’ve also cut down significantly on the number of reports that I have.)
For whatever it’s worth: I don’t recall wanting you to quit (as opposed to improve). I don’t recall feeling ill will towards you personally. I do not now think poorly of you personally on account of your efforts on the MIRI ops team.
As to the question of how these reports hit my ear: they sound to me like accurate recountings of real situations (in particular, I recall the bike pump one, and suspect that the others were also real events).
They also trigger a bunch of defensiveness in me. I think your descriptions are accurate, but that they’re missing various bits of context.
The fact that there was other context doesn’t make your experience any less shitty! I reiterate that I would have preferred it be not-at-all shitty.
Speaking from my sense of defensiveness, and adding in some of that additional context for the case that I remember clearly:
If memory serves: in that era, the ops team was experimenting with trying to make everyone’s lives easier by doing all sorts of extra stuff (I think they were even trying to figure out if they could do laundry), as seemed like a fine experiment to try.
In particular, I wasn’t going around being like “and also pump my bike tires up”; rather, the ops team was soliciting a bunch of little task items.
If memory serves: during that experiment, I was struggling a bunch with being told that things would be done by times, and then them not being done by those times (as is significantly worse than being told that those things won’t be done at all—I can do it myself, and will do it myself, if I’m not told that somebody else is about to do it!)
If memory serves: yep, it was pretty frustrating to blow a tire on a bike during a commute after being told that my bike tires were going to be inflated, both on account of the danger and on account of then having to walk the rest of the commute, buy a new tire, swap the tire out, etc.
My recollection of the thought that ran through my mind when you were like “Well I couldn’t figure out how to use a bike pump” was that this was some sideways attempt at begging pardon, without actually saying “oops” first, nor trying the obvious-to-me steps like “watch a youtube video” or “ask your manager if he knows how to inflate a bike tire”, nor noticing that the entire hypothesized time-save of somebody else inflating bike tires is wiped out by me having to give tutorials on it.
Was saying “well then learn!” and leaving a good solution, by my lights? Nope! Would I have been living up to my conversational ideals (significantly) better, if I’d said something like “Sorry, I don’t have capacity for this conversation, please have it with your manager instead” in a gentle tone? Yep!
I do have some general sense here that those aren’t emotionally realistic options for people with my emotional makeup.
I aspire to those sorts of reactions, and I sometimes even achieve them, now that I’m a handful of years older and have more practice and experience. But… still speaking from a place of defensiveness, I have a sense that there’s some sort of trap for people with my emotional makeup here. If you stay and try to express yourself despite experiencing strong feelings of frustration, you’re “almost yelling”. If you leave because you’re feeling a bunch of frustration and people say they don’t like talking to you while you’re feeling a bunch of frustration, you’re “storming out”.
Perhaps I’m missing some obvious third alternative here, that can be practically run while experiencing a bunch of frustration or exasperation. (If you know of one, I’d love to hear it.)
None of this is to say that your experience wasn’t shitty! I again apologize for that (with the caveat that I still don’t feel like I see practical changes to make to myself, beyond the only-partially-successful changes I’ve already made).
For the record, I 100% endorse you leaving an employment situation where you felt uncomfortable and bad (and agree with you that this is the labor market working-as-intended, and agree with you that me causing a decent fraction of employees to have a shitty time is an extra cost for me to pay when acting as an employer).
I have some replies to Nate’s reply.
Overview:
Once again, I’m not asking anyone to modify their personality at all. I mainly wish I had been warned about what was in store for me when I joined MIRI, and I want such warnings to be a visible part of Nate’s reputation.
I feel some pressure to match Nate’s conciliatory tone, but something feels incongruous about it. I’m concerned that people will read Nate’s calm, kindly replies and come away with the wrong idea of how he presented himself at MIRI.
I find Nate’s additional context to be, well…missing some important context. See below...
More context and recollections
It’s true that I didn’t report directly to Nate, and there could be a reasonable expectation that I refrain from bothering him without at least talking to my manager first. My memory is that this was a practical emergence, and not an explicit rule. Regardless, it seemed that Nate was sort of having it both ways, because he did in fact sometimes directly ask me questions (while quite angry), for example why we had ordered lunch from a restaurant he didn’t like, or where the soy sauce was. I now have to wonder what would have happened if I had refused to answer those angrily posed questions on the grounds that I didn’t report directly to him. My guess is that I probably would have lost the job shortly thereafter (and been happier for having held my boundaries—such a story would have been the labor market functioning even more efficiently).
I told the story about how he got very angry when I didn’t inflate his tires properly and one went flat during his commute. He added mitigating context claiming that he was rushed and sweaty having just replaced the tire, and then felt his own (implicit) boundaries violated by my approach.
Well, my memory is that I had his tire professionally replaced. Maybe I misremember that detail. But I am quite sure I took his bike to the shop for some kind of repair in the immediate wake of the flat tire. After that, I messaged him to let him know the repairs were finished and his bike was back on the office bike rack, ready to use. It wasn’t necessary to send him that message, but I wanted to do the small professional courtesy as a gesture of respect and de-escalation. I regret that, because he didn’t reply at all.
In the next day or so, we ended up taking the elevator at the same time. We stood in cold silence. I got no signal that he felt anything other than annoyance and disregard. I would have accepted an apology or just some kind of thawing, but I sure as hell wasn’t going to risk initiating another conversation he didn’t want. When he says he has no recollection of wanting me to quit (as opposed to improve), I feel frustrated because I think this will give people a distorted mental image of what his outward demeanor was like.
Alternate timelines that I would have preferred
Nate says he feels trapped:
Here are six alternatives, ranked by my preference. I personally think none of them seem impossible and all of them are reasonable to expect from an agentic, cooperative EA leader. Each scenario is meant to be imagined independently of the others.
In 2014 Nate writes his blog and crafts his public image in such a way that by the time I am recruited to work for MIRI, I have gotten a full and realistic sense of his emotional build, and quickly decline the job offer from MIRI (or at least bargain for modifications to the job description). This probably would have required placing several prominent disclaimers throughout the blog, as well as changing the tone and implication of many of the posts. Nate accepts the reputational costs that this modified narrative requires.
MIRI makes sure their ops recruitment checklist includes giving explicit warnings about Nate’s emotions, and I quickly decline the job offer with no hard feelings. This one seems the most realistic to me.
Nate firms up the organizational separation. Before having any angry outbursts, it is made highly explicit that I should not bring up e.g. bike tire questions with Nate, and furthermore that Nate should not e.g. ask me to explain perceived lunch errors. Those things are to be resolved with my manager first if possible. I am warned that he may not be able to calmly tolerate violations of these boundaries. It is acknowledged that this is kind of weird, but MIRI is a weird org trying to save the world.
If he had done this, I probably would have felt uneasy, still lost some respect for him, and told some friends about it. I probably still would have wanted to quit, and felt some mild skepticism toward the org ever after. But I would have felt much better about it, and I wouldn’t feel the need now to warn people that Nate was given unadvertised exemptions from social norms.
During the bike pump incident, Nate incurs an extra 30 seconds of wasted time, and hopefully no permanent emotional damage, in order to pad the angry interruption (“Learn!”) with a few extra syllables. Not even especially polite syllables, just enough to signal that he knows about and complies with the shared burden of emotional regulation. Something like, “Okay, I’m going cut you off right there–all I’m going to say is that I’m really unhappy and can’t deal with this right now. You should talk about this with your manager.”
Not long after the bike pump incident, Nate and I end up taking the elevator at the same time. He takes the opportunity to give at least a perfunctory apology and explain why he snapped. I get the sense that he genuinely regrets not establishing boundaries sooner. I also get the sense that he does not actively want me to quit (presuming that is actually true).
Nate doesn’t initiate any apologies, but he does send some kind of signal that things won’t go wrong if I ever try to bring it up again. He does not continue signaling a hair trigger while walking around the office. When I message him to let him know that the bike is back, he replies with something at least as cordial as “ok good to know”.
So I don’t think there’s any reasonable sense in which Nate was “trapped”.
What I want right now
I’m not especially bidding for him to change his emotional habits. In fact, I don’t really want much of anything from Nate himself.
When I called up my former trainee the other day, I did not say, “I’m sorry Nate was angry and toxic to you.” Rather, I said, “I’m sorry I didn’t make absolutely sure you knew what you were getting into. I’m sorry for not giving you the thorough disclaimer that I wish someone had given me.”
What I do really want is for Nate’s reputation to catch up to his behavior. I think this is already happening to some extent, and the community is appropriately deducting some prestige and bargaining power.
Some people I respect have already told me that they now think significantly differently about Nate, which I think is right and proper, as well as being some relief to me personally.
Looking Forward
I hope that if there is any more that ought to be revealed, that it is. I hope that bright-eyed young people entering the scene will be warned about him as thoroughly as I was warned about Eliezer (or more). I also hope that Nate does not suffer any unfair or unreasonable consequences[1]–I admit that I would feel some momentary satisfaction from that sort of thing, but it wouldn’t be worth diluting the valid complaints. Humans have an intrinsic drive for scapegoating, or so I have heard, and that’s the kind of overindulgence that is not affordable in the fight for the future.
I look forward to less often hearing Nate’s persona invoked, with no caveats, as an exemplar of agency and self-understanding. He has some notable virtues, and has done things worthy of commendation, but I hope that the halo effect around him substantially diminishes.
This risks derailment, but I hope that some new AI notkilleveryoneism researchers manage to loosen Nate’s monopoly on his niche at MIRI. By that I mean that it would be cool if, by some miracle, Eliezer found enough energy that he could invest some of it in a few final, dignified hail-mary apprenticeships. Instead of hearing that Eliezer and Nate said something, I hope I get to hear that The Cool New Eliezer-Approved AI notkilleveryoneism Team said something. I’m not holding my breath but that’s my hope.
I have some skepticism toward all of Nate’s self-reports, including the ones about his recent improvements. I am generally pessimistic about adults changing very much, especially when they are in their thirties or older. But...if Nate beats the odds in a hard-to-fake way, I will be duly surprised and impressed.
More miscellaneous notes
(Misc but still important)
I feel some pressure right now to match Nate’s conciliatory tone–pressure to realize that Nate was just a cooperative guy all along who honestly wanted the best and fell short of his own values. I also feel an opposite desire to shout, “No! He’s predictably performing gentleness now that he’s at a disadvantage! Onlookers will overweight this and walk away with the wrong overall impression!!!” I endorse voicing that feeling. I find Nate’s replies here just a bit too slippery, and I urge people to treat this as an occasion where actions speak louder than words. Questioning people’s motives is often considered bad form, so I will simply say that I believe that Nate is not outright lying about his thoughts or feelings, but also that Sarah Constantin’s Player vs Character model is weighing on my mind.
I want it to be known that during Nate’s outbursts, I was unsure about where his actual limits were. I obviously never worried that he would, like, punch me or something, since that’s quite illegal. But let me put it this way: If Rob had started shouting at me, I would have been utterly shocked, whereas if Nate had done that, I would have been merely surprised. Regardless of what Nate claims about his internal states, I think it’s not a coincidence that my uncertainty about potential escalation was so large, and that I was stressed into doing more emotional labor than I signed on for. To say that differently: I think it’s not a coincidence that his actions reliably gave him the benefits of brinkmanship. I find it entirely plausible that if Nate’s incentives had been different, then his emotional impulses would have adapted. But again, I would have been fine with just being forewarned during recruitment.
There were other highly regarded researchers at MIRI whose lunch orders I messed up, and who are not known for empathy or people skills, but who reliably managed to express mild disappointment instead of hangry hostility.
Nate said:
”My recollection of the thought that ran through my mind when you were like ‘Well I couldn’t figure out how to use a bike pump’ was that this was some sideways attempt at begging pardon, without actually saying ‘oops’ first, nor trying the obvious-to-me steps like ‘watch a youtube video’ or ‘ask your manager if he knows how to inflate a bike tire’[...]”
Nate has a point here—it’s entirely plausible that the particular bike-related anger could have been resolved on my end the way he suggests. But I also find it entirely plausible that the difficulty could have persisted even after some googling and YouTubing, and I have to wonder what would have happened in that case. If I had managed to say the word “YouTube” early enough in that interaction, would Nate have let me finish my sentence? Well, that’s possible, and there are other possibilities too. So maybe Nate feels like he was tracking things cleanly enough that no one would trigger his rage unless they definitely had a serious lapse in their responsibilities (which is what most people think of themselves). But personally I still doubt that.
“[...] the entire hypothesized time-save of somebody else inflating bike tires is wiped out by me having to give tutorials on it.” This seems like an overstatement and I want to defend my common sense. I continue to think that the trouble I was having with the pressure-checking might possibly have been entirely solved by having me come watch him do it once.
The quotation, “Well, I couldn’t figure out how to use a bike pump” is a misrepresentation of what happened, and I’m disappointed to find myself defending against it. I had owned and maintained multiple bikes without running into this problem. Different equipment sometimes has subtleties. I still don’t know what was going wrong, and indeed it might have been a silly error on my part, but I claim that I was not as stupid, incompetent, or lazy as Nate makes me sound here.
It’s very dramatic, but I have to say I understand why TurnTrout said he would burn 10% of his liquidity in exchange for never having a certain conversation with Nate. My own experiences with angry Nate were weirdly unsettling. Again, I’ve endured objectively harsher treatment from superiors and customers in previous jobs, but those didn’t feel as bad. I think the difference had something to do with everyone I respected praising him, nuancing away his flaws, leaving me wondering what the hell was going on.
Alex Turner (TurnTrout) is my friend, we regularly talk and hang out and get food. In real life I think Alex frequently shows more pro-sociality and self-control than I do. When I think of Alex taking that kind of psychic damage from Nate, I feel…just…intolerably scandalized, and need to do deliberate mental tricks to shake it off.
Some positivity
I’m really happy overall with the replies I received from everyone. I expected to encounter a lot more resistance and skepticism, and I wouldn’t have blamed people for it. I have no regrets about anything I wrote. Without identifying anyone, let me say I’m also happy with what people have told me in private. It’s quite nice to have it confirmed that I’m not crazy and that I’ve helped make things better. And to those of you who have taken decisive actions: I honestly consider you to be heroes.
Even though it comes quite late, I’m glad to hear that there is some kind of effort being made to give people the warning I wish I had received. It’s maybe also good that Nate stopped being the executive director.
I sat on my complaints for 5+ years. But then last week, Alex Turner said the bravest public and true thing about Nate I had yet seen: that people are failing to enforce norms on him. Shortly thereafter, I overheard some people I respect having an actually nuanced conversation about it, not just the spurious, unbalanced “nuance” that had been protecting the status quo for years. Alex’s lonely dissent and the reverberations it caused inspired me to break my silence. Hero.
My experiences elsewhere in the ecosystem have been much better. I’ve been at my current job for over 18 months, and have witnessed none of the problems that Nate had while I was at MIRI. My coworkers clearly put in the effort every day to cooperate, and I feel good about the future of the org.
Some Links
Here are some links that have affected how I think about human affairs in general, especially interpersonal drama. If anyone finds my takes here to be too cynical, then these links will at least show where I picked up a lot of that cynicism. (However, I also fully agree with the cynical about cynicism post.)
Corrupted Hardware
Player vs Character
Algorithmic Intent
[1] It was pointed out to me that no one hopes for unfair or unreasonable consequences, so my words here are vague. Yeah okay. People can debate what’s fair and reasonable, and I may join the debate or not.
Oct 11 & 12 EDITS: Restructured some sentences, swapped out some words, and added a whole bullet point to the Misc section.
Here’s my take: From the inside, Nate feels like he is incapable of not becoming very frustrated, even angry. In a sense this is true. But this state of affairs is in fact a consequence of Nate not being subject to the same rules as everybody else.
I think I know what it’s like, to an extent — I’ve had anger issues since I was born, and despite speaking openly about it to many people, I’ve never met anyone who’s been able to really understand the feeling of being overwhelmed with rage (especially not as an adult). That feeling can be very hard to control.
However, I am constantly aware that having an angry outburst is massively socially unacceptable, to the point where if I let such things happen regularly I would lose my job / my standing in the community / all my friends / everyone close to me. This creates an extremely strong incentive for me to self-regulate at least my outward reactions, even when it’s really hard. But because Nate is so high-status, he is allowed to make such outbursts without being faced with losing his job, his standing in the community, or his friends. This means he is insufficiently incentivized to self-regulate, and thus has been unable to learn.
I think it’s cool that you’re engaging with criticism and acknowledging the harm that happened as a result of your struggles.
And, to cut to the painful part, that’s about the only positive thing that I (random person on the internet) have to say about what you just wrote.
In particular, you sound (and sorry if I’m making any wrong assumption here) extremely unwilling to entertain the idea that you were wrong, or that any potential improvement might need to come from you.
You say:
But you don’t seem to consider the idea that maybe you were more in a position to improve than he was.
I don’t want to be overly harsh or judgmental. You (eventually) apologize and acknowledge your responsibility in employees having a shitty time, and it’s easy for an internet stranger to over-analyze everything you said.
But. I do feel confident that you’re expressing a lack of curiosity here. You’re assuming that there’s nothing you possibly have done to make Kurt’s experience better, and while you’re open to hearing if anyone presents you with a third option, you don’t seem to think seeking out a third option is a problem you should actively solve.
Like, here… You get that you’re not really engaging with what Kurt is/was saying, right?
Kurt’s point is that your pump seemed harder to use than other bike pumps. If the issue is on the object level, valid answers could be asking what types of bike pumps he’s used to and where the discrepancy could come from, suggesting he buy a new pump, or if you’re feeling especially curious asking that he bring his own pump to work so you can compare the two; or maybe the issue could come not from the pump but from the tires, in which case you could consider changing them, etc.
If the issue is on the meta level and that you don’t want to spend time on these problems, a valid answer could be saying “Okay, what do you need to solve this problem without my input?”. Then it could be a discussion about discretionary budget, about the amount of initiative you expect him to have with his job, about asking why he didn’t feel comfortable making these buying decisions right away, etc.
Your only takeaway from this issue was “he was wrong and he could have obviously solved it watching a 5 minutes youtube tutorial, what would have been the most efficient way to communicate to him that he was wrong?”. At no point in this reply are you considering (out loud, at least) that hypothesis “maybe I was wrong and I missed something”.
Like, I get having a hot temper and saying things you regret because you don’t see any other answers in the moment. But part of the process is to communicate despite a hot temper is to be willing to admit you were wrong.
The best life-hack I have is “Don’t be afraid to come back and restart the discussion once you feel less frustration or exasperation”.
Long-term, I’d recommend looking into Non-Violent Communication, if you haven’t already. There’s a lot of cruft in there, but in my experience the core insights work: express vulnerability, focus on communicating you needs and how you feel about things, avoid assigning blame, make negotiable requests, and go from there.
So for the bike tire thing the NVC version would be something like “I need to spend my time efficiently and not have to worry about logistics; when you tell me you’re having problems with the pump I feel stressed because I feel like I’m spending time I should spend on more important things. I need you to find a system where you can solve these problems without my input. What do you need to make that happen?”
I’ve been dating Nate for two years (tho wanna clarify we are not doing marriage-kids and we’re both actively looking for more serious other partners).
Nate is profoundly wonderful in many ways, like often surprises me in new ways of wonderfulness, and has raised my standards in partners. He’s deeply caring, attentive, competent, hilarious, and of course brilliant.
Also, many of the complaints about him in the comments resonate with my experience, particularly your description above. I often find that in disputes I feel dismissed, I perceive him as having a significant lack of curiosity about my worldview (and believe he’s explicitly said he’s not curious about perspectives he anticipates to have no value to him).
Iirc he’s explicitly said he doesn’t respect my thinking (edit: he clarifies he respects it in some areas but not others), and from my perspective this radiates off him whenever we fight. I often feel like I have trouble trusting my own mind, I doubt myself, and despite my best attempts I somehow come out of disputes thinking I must be the one who’s wrong. It’s weird to have a partner who’s so shockingly good in so many ways, yet we have maybe the worst fights I’ve ever experienced in romantic relationships. (Though he says other girls he’s dated don’t have this problem and I am unusual)
On one plus side, I’ve found him to be very good at installing concrete changes if you can articulate them to him. A few times I managed to have a specific request about how I’d like him to say things differently, and if he agrees to do so he updates fast, thoroughly, and permanently.
I feel conflicted about posting this here because ??? should this be personal/private ?? but I’m having some sort of massive relief and feeling like actually I’m not insane. And also I am invested in (though not hopeful about) something changing here cause it would be good for our relationship and I assume also MIRI, which I like and believe in.
(I talked to Nate before posting this comment and he was encouraging)
Thanks <3
(To be clear: I think that at least one other of my past long-term/serious romantic partners would say “of all romantic conflicts, I felt shittiest during ours”. The thing that I don’t recall other long-term/serious romantic partners reporting is the sense of inability to trust their own mind or self during disputes. (It’s plausible to me that some have felt it and not told me.))
Chiming in to provide additional datapoints. (Apologies for this being quite late to the conversation; I frequent The Other Forum regularly, and LW much less so, and only recently read this post/comments.) My experience has been quite different to a lot of the experiences described here, and I was very surprised when reading.
I read all of the people who have had (very) negative experiences as being sincere and reporting events and emotions as they experienced them. I could feel what I perceived to be real distress and pain in a lot of the comments, and this was pretty saddening.
Note: I really don’t like posting personal information on the public internet, for both personal preference and professional reasons. (I felt sure I wanted to post this, though.)
Background: I dated Nate on-and-off for ~6 years (from 2016-2022). We’re now friends on good terms.
How I experienced Nate’s communication over the years:
During disputes, I felt Nate respected my views and my feelings. I felt Nate tried to communicate well with me (though of course this is often hard for both parties in the midst of a dispute and we both failed a bunch).
During disputes, and generally, I felt like Nate engaged with me as an equal. I don’t recall ever feeling dismissed in the ‘you’re not as smart as me’ vein (or any other vein).
It feels important to note that we never really had discussions about his work in the way a lot of people in the comments seemed to have had (where they had very bad experiences). We talked about our respective work a bunch, but the subject matter wasn’t technical (I don’t have a technical background).
I don’t recall Nate losing his cool/saying something mean, except for uncommon instances where I was very upset and was saying unkind things first (not my proudest moments!)
I ~never felt like Nate’s confidence/manner impacted my views in a way I don’t endorse or made me feel less sure of my own beliefs. Nate is certainly intense and confident-sounding, but for whatever reason, I don’t recall this in particular causing issues.
The ways in which our communication was bad felt to me more mundane and predictable as relationship-comms go (unclear communications around expectations, not recognising and addressing underlying issues/feelings quickly enough, etc). And none of this was the reason we ultimately parted ways.
Also, in my experience, Nate’s communication / handling of disputes has improved over time.
I don’t want to paint an overly rosy picture—our worst disputes were really not great (understatement to save me saying personal things), but for reasons basically unrelated to the themes of the comments here. Nate is flawed in loads of ways (I was always pretty icked-out at the reverence? people treated him with, like he was perfect or something); and though I haven’t had the experiences others here have had, I find him to be someone who wants to be better and knows he’s far from perfect.
Perhaps you’re trying to point at something that I’m missing, but from my point of view, sentences like “I’d love to say “and I’ve identified the source of the problem and successfully addressed it”, but I don’t think I have” and “would I have been living up to my conversational ideals (significantly) better, if I’d said [...]” are intended indicators that I believe there’s significant room for me to improve, and that I have desire to improve.
At to be clear: I think that there is significant room for improvement for me here, and I desire to improve.
(And for the record: I have put a decent amount of effort towards improving, with some success.)
(And for the record: I don’t recall any instances of getting frustrated-in-the-way-that-turntrout-and-KurtB-are-recounting with Thomas Kwa, or any of Vivek’s team, as I think is a decent amount of evidence about those improvements, given how much time I spent working with them. (Which isn’t to say they didn’t have other discomforts!))
This reply wouldn’t have quite suited me, because Kurt didn’t report to me, and (if memory serves) we’d already been having some issues of the form “can you solve this by using your own initiative, or by spending modest amounts of money”. And (if memory serves) I had already tried to communicate that these weren’t the sorts of conversations I wanted to be having.
(I totally agree that his manager should have had a discussion about discretionary budget and initiative, and to probe why he didn’t feel comfortable making those buying decisions right away. He was not my direct report.)
Like, the context (if I recall correctly, which I might not at 6ish years remove) wasn’t that I called Kurt to ask him what had happened, nor that we were having some sort of general meeting in which he brought up this point. (Again: he didn’t report to me.) The context is that I was already late from walking my commute, sweaty from changing a bike tire, and Kurt came up and was like “Hey, sorry to hear your tire popped. I couldn’t figure out how to use your pump”, in a tone that parsed to me as someone begging pardon and indicating that he was about to ask me how to use one, a conversation that I did not want to be in at that moment and that seemed to me like a new instance of a repeating issue.
Nope!
I did (and still do) believe that this was an indication that Kurt wasn’t up to the challenge that the ops team was (at that time) undertaking, of seeing if they could make people’s lives easier by doing annoying little tasks for them.
It’s not obvious to me that he could have solved it with a 5 minute youtube tutorial; for all I know it would have taken him hours.
(Where the argument here is not “hours of his time are worth minutes of mine”; I don’t really think in those terms despite how everyone else seems to want to; I’d think more in terms of “training initiative” and “testing the hypothesis that the ops team can cheaply make people’s lives better by handling a bunch of annoying tasks (and, if so, getting a sense for how expensive it is so that we can decide whether it’s within budget)”.)
(Note that I would have considered it totally reasonable and fine for him to go to his manager and say “so, we’re not doing this, it’s too much effort and too low priority”, such that the ops team could tell me “X won’t be done” instead of falsely telling me “X will be done by time Y”, as I was eventually begging them to do.)
My takeaway wasn’t so much “he was wrong” as “something clearly wasn’t working about the requests that he use his own initative / money / his manager, as a resource while trying to help make people’s lives easier by doing a bunch of little tasks for them”. Which conclusion I still think I was licensed to draw, from that particular interaction.
oh absolutely not, “well then learn!” is not a calculated “efficient” communication, it’s an exasperated outburst, of the sort that is unvirtuous by my conversational standards.
As stated, “Sorry, I don’t have capacity for this conversation, please have it with your manager instead” in a gentle tone would have lived up to my own conversational virtues significantly better.
I’m still not really considering this hypothesis (even internally).
This “X was wrong” concept isn’t even a recognizable concept in my native cognitive format. I readily believe things like “the exasperated outburst wasn’t kind” and “I would have lived up to my conversational virtues more if I had instead been kind” and “it’s worth changing my behavior to live up to those virtues better”. And I readily believe things like “if Kurt had taken initiative there, that would have been favorable evidence about his ability to fill the role he was hired for” and “the fact that Kurt came to me in that situation rather than taking initiative or going to his manager, despite previous attempts to cause him to take initiative and/or go through his manager, was evidence against his ability to fill the role he was hired for”.
Which you perhaps would parse as “Nate believed that both parties Were Wrong”, but that’s not the way that I dice things up, internally.
Perhaps I’m being dense, and some additional kernel of doubt is being asked of me here. If so, I’d appreciate attempts to spell it out like I’m a total idiot.
Thanks! “Circle back around after I’ve cooled down” is indeed one of the various techniques that I have adopted (and that I file under partially-successful changes).
Thanks again! (I have read that book, and made changes on account of it that I also file under partial-successes.)
If memory serves, the NVC book contains a case where the author is like “You can use NVC even when you’re in a lot of emotional distress! For instance, one time when I was overwhelmed to the point of emotional outburst, I outburst “I am feeling pain!” and left the room, as was an instance of adhering to the NVC issues even in a context where emotions were running high”.
This feels more like the sort of thing that is emotionally-plausible to me in realtime when I am frustrated in that way. I agree that outbursts “I’m feeling frustrated” or “I’m feeling exasperated” would have been better outbursts than “Well then learn”, before exiting. That’s the sort of thing I manage to hit sometimes with partial success.
And, to be clear, I also aspire to higher-grade responses like a chill “hey man, sorry to interrupt (but I’m already late to a bunch of things today), is this a case where you should be using your own initiative and/or talking to your manager instead of me?”. And perhaps we’ll get there! And maybe further discussions like this one will help me gain new techniques towards that end, which I’d greatly appreciate.
So I’ve been thinking about this particular branch for a while and I think I have a slightly different diagnosis from PoignardAzur, which I think nearly lines up with yours but has an important difference. I think this is the important part:
Even if you are not tracking who is Wrong is any particular interaction, if other people are tracking who is Wrong, that seems like an important thing for you to handle because it will be a large part of how they interpret communication from you. (For the bike pump example, the thing where you saw Kurt as “begging pardon” seems like evidence this was plausibly up for Kurt / you could have guessed this was up for Kurt in the moment.) One way to interpret the situation is:
Kurt: I am Wrong but would like to displace that to the bike pump
Nate: Rejected! >:[
Kurt: :(
I am imagining that you were not asking for this sort of situation (and would have been less interested in a “save your time” deal if “do emotional labor for people helping you” had explicitly been part of the deal), but my guess is attention to this sort of thing is the next place to look for attacking the source of the problem.
[Also, I’m not trying to confidently assert this is what was actually up for Kurt in the moment—instead I’m asking “if this story made me side with Kurt, why did that happen?”]
I don’t know if “dense” is the word I use, but yeah, I think you missed my point.
My ELI5 would be “You’re still assuming the problem was ‘Kurt didn’t know how to use a pump’ and not ‘there was something wrong with your pump’”.
I don’t want to speculate too much beyond that eg about the discretionary budget stuff.
Happy to hear that!
(I had used that pump that very day, shortly before, to pump up the replacement tire.)
I talked to Kurt in some detail. Nate never apologized or acknowledged the bike pump incident (until now). After that incident, Nate never came back and said e.g. “wow, I was really frustrated earlier, sorry for taking that out on you!” The next time Kurt was alone with him was in the elevator later that week, and there was a cold silence that neither of them broke.
One alternative could be to regulate your emotions so you don’t feel as intense frustration from a given epistemic position? I think this is what most people do.
I suspect that lines like this are giving people the impression that you [Nate] don’t think there are (realistic) things that you can improve, or that you’ve “given up”.
My understanding is that your perspective is something like “I feel like I recognize that there’s stuff I can do to improve, and I’ve tried to put a lot of energy into finding those improvements, and I’m pretty open to others suggesting specific things I could do. But a lot of things that other people think would be easy fixes actually don’t work or don’t work for someone with my emotional makeup (e.g., because they’re super costly or because I don’t end up being able to implement them well.)
Like, my guess is that some people are like “wait what, why is it costly or infeasible for you to just gently tell someone that you don’t have time/energy for a conversation and politely tell them to talk to their manager?”
And then their reaction is “oh, Nate must not care or must not be trying.” (Which, to be clear, I find to be a reasonable hypothesis. Or at least the continuous version, which is something like “hm, maybe Nate doesn’t care enough or isn’t trying as much as I think the community should expect people in his position to try” or something.)
And then your reaction is “gosh, I am trying, but people seem to systematically underestimate how costly/infeasible a lot of their suggestions are.”
I’m not really sure what to do about this, other than “maybe if this is acknowledged, it’ll lead to more a more constructive dialogue between you and those who want you or expect you to change.”
How do you/MIRI communicate about working with you now?
Insofar as you’re querying the near future: I’m not currently attempting work collaborations with any new folk, and so the matter is somewhat up in the air. (I recently asked Malo to consider a MIRI-policy of ensuring all new employees who might interact with me get some sort of list of warnings / disclaimers / affordances / notes.)
Insofar as you’re querying the recent past: There aren’t many recent cases to draw from. This comment has some words about how things went with Vivek’s hires. The other recent hires that I recall both (a) weren’t hired to do research with me, and (b) mentioned that they’d read my communication handbook (as includes the affordance-list and the failure-modes section, which I consider to be the critcial pieces of warning), which I considered sufficient. (But then I did have communication difficulties with one of them (of the “despair” variety), which updated me somewhat.)
Insofar as you’re querying about even light or tangential working relationships (like people asking my take on a whiteboard when I’m walking past), currently I don’t issue any warnings in those cases, and am not convinced that they’d be warranted.
To be clear: I’m not currently personally sold on the hypothesis that I owe people a bunch of warnings. I think of them as more of a sensible thing to do; it’d be lovely if everyone was building explicit models of their conversational failure-modes and proactively sharing them, and I’m a be-the-change-you-wanna-see-in-the-world sort of guy.
(Perhaps by the end of this whole conversation I will be sold on that hypothesis! I’ve updated in that direction over the past couple days.)
(To state the obvious: I endorse MIRI institutionally acting according to others’ conclusions on that matter rather than on mine, hence asking Malo to consider it independently.)
One frame I want to lay out is that it seems like you’re not accounting for the organizational cost of how you treat employees/collaborators. An executive director needing to mostly not talk to people, and shaping hiring around social pain tolerance, is a five alarm fire for organizations as small as MIRI. Based on the info here, my first thought is you should be in a different role, so that you have fewer interactions and less implied power. That requires someone to replace you as ED, and I don’t know if there are any options available, but at a minimum I think you/MIRI should be treating the status quo as potentially extremely costly, and taking steps to assess the total cost and potential fixes.
I could be wrong here, 98% of my information is from this post + comments, but I get the sense you/MIRI haven’t looked sufficiently hard to even assess what the costs are. It sounds like you have asked people, which is great and more than most orgs do, but I get the sense you haven’t grappled with the magnitude of the costs beyond the personal and social.
Nate stepped down as ED shortly after [edit: actually before] our project ended, the website just hasn’t been updated. I’m not sure what exactly the organizational structure is now, but you can probably message @lisathiergart for an update.
Edit: there is now an announcement.
This comment’s updates for me personally:
The overall “EA is scary / criticizing leaders is scary” meme is very frequently something I roll my eyes at, I find it alien and sometimes laughable when people say they’re worried about being bold and brave cuz all I ever see are people being rewarded for constructive criticism. But man, I feel like if I didn’t know about some of this stuff then I’m missing a huge piece of the puzzle. Unclear yet what I’ll think about, say, the anon meta on forums after this comment sinks in / propagates, but my guess is it’ll be very different than what I thought before.
People are way too quick to reward themselves for trying (my update is my priority queue in doing a proper writeup): Nate & enablers saying that productivity / irreplaceability is an excuse to triage out fundamental interpersonal effort is equivalent (as far as I’m concerned) to a 2022 University Community Builder (TM) deciding that they’re entitled to opulent retreats the moment they declare stated interest in saving the world. “For the greater good” thinking is fraught and dicey even when you’re definitely valuable enough for the case to genuinely be made, but obviously there’s pressure toward accepting a huge error rate if you simply want to believe you or a colleague is that productive/insightful. I honestly think Nate’s position here is more excusable than enablers: you basically need to see nobel physicist level output before you consider giving someone this much benefit of the doubt, and even then you should decide not to after considering it, I’m kinda dumbfounded that it was this easy for MIRI’s culture to be like this. (yes my epistemic position is going to be wrong about the stakes because “undisclosed by default”, but there are a bajillion sources of my roll to disbelieve if anyone says “well actually undisclosed MIRI codebases are nobel physicist level).
I feel very vindicated having written this comment, and I am subtracting karma from everyone who gave Nate points for writing a long introspective gdoc. You guys should’ve assumed that it would be a steep misfire.
Someone told me that some friends of theirs hated a talk or office hours with Nate, and I super devil’s advocated the idea that lots of people have reasons for disliking the “blunt because if I suffer fools we’ll all lower our standards” style that I’m not sympathetic with: I now need to apologize to them for being dismissive. I mean for chrissakes yall, in my first 1:1 with Eliezer he was not suffering fools, he helped me speedrun noticing how misled my optimism about my project at the time was and it was jovial and pleasant, so I felt like an idiot and I look back fondly on the interaction. So no, the comments about how comms style is downstream of trying to outperform those prestigious etiquette professional academics goodharting on useless but legible research that Nate retreats to elsewhere in the comments here do not hold.
Extremely from the heart warm comments about Nate from my PoV (not coming from a phonedin/trite/etiquette “soften the blow” place, but very glad that there’s that upside):
I’m a huge replacing guilt fan
reading Nate on github and lesswrong has been very important to me in my CS education. The old intelligence.org/research-guide mattered so much to me at very important life/development pivot.
Nate’s strategy / philosophy of alignment posts, particularly recently, have been phenomenal.
in a sibling comment, Nate wrote:
This is hard and unfair and I absolutely feel for him, I’ve been there[1].
I don’t know if we’ve ever been in the same room. I’m going off of web presence, and very little comments or rumors others have said.
on second thought: I’ve mostly only been there in say a soup kitchen run by trans commie lesbians, who are eagerly looking for the first excuse they can find to cancel the cis guy. I guess I don’t at all relate to the possibility that someone would feel that way in bay area tech scene.
This is a generally constructive comment. One bit left me confused, and I wonder if you can unpack what it means?
What was the misfire? (I mean literally what does ‘it’ stand for in this sentence?) Also, what kind of points and karma are we talking about, presumably metaphorical?
https://www.lesswrong.com/posts/BGLu3iCGjjcSaeeBG/related-discussion-from-thomas-kwa-s-miri-research?commentId=fPz6jxjybp4Zmn2CK This brief subthread can be read as “giving nate points for trying” and is too credulous about if “introspection” actually works—my wild background guess is that roughly 60% of the time “introspection” is more “elaborate self-delusion” than working as intended, and there are times when someone saying “no but I’m trying really hard to be good at it” drives that probability up instead of down. I didn’t think this was one of those times before reading Kurt’s comment. A more charitable view is that this prickliness (understatement) is something that’s getting triage’d out / deprioritized, not gymnastically dodged, but I think it’s unreasonable to ask people to pay attention to the difference.
That’s besides the point: the “it” was just the gdoc. “it would be a steep misfire” would mean “the gdoc tries to talk about the situation and totally does not address what matters”. The subtraction of karma was metaphorical (I don’t think I even officially voted on lesswrong!). I want to emphasize that I’m still very weak, cuz for instance I can expect people in that subthread to later tell me a detailed inside view about how giving Nate points for trying (by writing that doc) doesn’t literally mean that they were drawn into this “if von neumann has to scream at me to be productive, then it would be selfish to set a personal boundary” take, but I think it’s reasonable for me to be suspicious and cautious and look for more evidence that people would not fall for this class of “holding some people to different standards for for-the-greater-good reasons” again.
Separately, a friend pointed out that an important part of apologies is the doer showing they understand the damage done, and the person hurt feeling heard, which I don’t think I’ve done much of above. An attempt:
I hear you as saying that you felt a strong sense of disapproval from me; that I was unpredictable in my frustration as kept you feeling (perhaps) regularly on-edge and stressed; that you felt I lacked interest in your efforts or attention for you; and perhaps that this was particularly disorienting given the impression you had of me both from my in-person writing and from private textual communication about unrelated issues. Plus that you had additional stress from uncertainty about whether talking about your apprehension was OK, given your belief (and the belief of your friends) that perhaps my work was important and you wouldn’t want to disrupt it.
This sounds demoralizing, and like it sucks.
I think it might be helpful for me to gain this understanding (as, e.g., might make certain harms more emotionally-salient in ways that make some of my updates sink deeper). I don’t think I understand very deeply how you felt. I have some guesses, but strongly expect I’m missing a bunch of important aspects of your experience. I’d be interested to hear more (publicly or privately) about it and could keep showing my (mis)understanding as my model improves, if you’d like (though also I do not consider you to owe me any engagement; no pressure).
(quick mod note confirming this is Kurt Brown who worked at MIRI)
I have only met Eliezer once for about ~60 minutes, but I had the same experience. We talked in a group about alignment, and even though he ended up repeating many concepts he had already written about extensively online, he failed to explain those concepts condescendingly at all, which is not what I’ve come to expect the median person to do in that situation. It just seemed like he really wanted us to understand the problem.
I was sort of unsurprised at the mismatch between perception and reality, frankly, because Eliezer is a very awkward, sorta funny looking dude. In this vein I will note that I was mildly disappointed a few years back when I checked out MIRI’s team page and saw that (IMO), with the exception of the founder, all of the most attractive people were the ones in leadership positions. In my experience, in very relatively nerdy environments, people who look and sound not-like-nerds get social leeway to be domineering and dismissive if they choose. That might explain part of what happened with Nate, though I am not a bay resident and have virtually no inside info here.
This updated me, thank you. A fair amount, from “IDK, this sounds like it’s fairly likely to mainly be just people being sensitive about blunt confrontational communication in a context where blunt confrontational communication is called for” to “Maybe that, but sure sounds a lot like Nate has a general disregard for fellows—maybe there’s some internal story he has where his behavior would make sense if other people shared that story, but they don’t and that should be obvious and he should have not behaved that way given that they don’t”.
I’ll make a tentative observation: it seems that you’re still being euphemistic and (as you kind of note yourself) you’re still self-censoring a bit.
The words that you say are “he’s mean and scary” and “he was not subject to the same behavioral regulation norms as everyone else”. The words I would have said, given your description and his answer below is “he acts like an asshole and gets away with it because people enable him”.
I’ve known bosses that were mean and scary, but otherwise felt fair and like they made the best of a tough situation. That’s not what you’re describing. Maybe Nate is an amazing person in other ways, and amazingly competent in ways that make him important to work with, but. He sounds like a person with extremely unpleasant behavior.
Wanted to briefly add a perspective I didn’t see mentioned yet --
First—seems like you had a particularly rough interaction, and I do want to express empathy for that. I feel like I recognise some of the things you point to, and think it’s plausible that I might have been similarly demoralised by that situation, and that would really suck for me and I’d be really sad. So, genuinely sorry about that. I hope you’ll find ways to regain motivation that was unfairly lost, and the ability to draw on insights that ended up involuntarily screened off from you.
Second, the perspective I’ve come to hold for these situations is… Basically the world does seem full of people who are extraordinarily productive in important ways, and who also… are kind of d*cks. (Important footnote: [1])
As such:
I think exceptional people are often sufficiently rare that, as things go, I’d rather take a bunch of productive d*cks than tune down their cognitive spikiness at the cost of mulling the productive peaks
I observe and am strategic about how I allocate my soul and motivation points to things. In the past I would kind of always pour full soul into things, but that led to a lot of sadness, because other people by default might not be able to hold things that are precious to me, and if I unilaterally pour it on them, they also really don’t have a responsibility to hold it! Ouch.
I try to satisfy different needs from different people. In various professional domains I’ll be pretty thick skinned and put up with a lot of nonsense to extract interesting insights from people or get things done. Then with my partner or close friends I’ll do a bunch of stuff that’s emotionally nurturing and care a lot about holding aspects of each other’s experience in ways that aren’t rude.
I beware of people employing dynamics that get inside and mess with my OODA loop, and have various allergic responses to this, and might often explicitly limit interactions, or hold the interaction in a particular way.[2] Regardless of whether they’ve advertised being unusual in this regard, I just kind of have a way of holding my guard up
I think holding this stance is my best strategy for getting around. Man, sometimes you gain so much great stuff from people who are rude, or weird, or norm-violating in various other ways, and I think “developing your own set of personal strategies that allow you to put up with stuff” can be a pretty decent superpower, judiciously deployed.
You say:
So in light of the above, the way I orient to this would be something like: if someone is really killing it in terms of intellectual insight, or just getting important shit done—that’s the primary thing I care about around here (on LessWrong and the broader ecosystem). I’ll try hard to carve a space for them to get those things out. If they’re also a d*ck, I’ll proably avoid inviting them to friendly gatherings I organise, and I might even just not have them work closely on my team specifically, because it’ll mess too much with my OODA loop, and I want a certain culture.
But I do not think they have a responsibility to proactively inform people about their style.
On a community-wide level, the ratio of reward I’d give out for insight/productivity vs punishment for rudeness is like at least 30:1 or something, on some imaginary scale. I don’t like rudeness and work best among people who are pretty empathetic and nurturing; but hey, the world is what it is, I’ll take what I can get, and think this is the best point on the tradeoff curve.
(And again, to reiterate, I do hope you also have or will find a way to orient to these things where you can gain the good insights + motivation, and avoid taking annoying hit points!)
Note: I don’t want to make any strong claims here about how insightful, or how much of a d*ck, this one particular empirical guy Nate is, with whom I have only interacted very little (though I like his blog posts!). Don’t construe my comment as claiming that he is actually either of those things!
Though I do find OODA loop fuckery and frame control much worse than “rudeness”
I tried to communicate “we should indeed subtract points for people being rude and aggressive” and “stop double-counting evidence by reminding yourself that someone might also be productive; that’s already factored into your assessment of them.”
It seems like you’re saying “I can imagine many cases where rude people have net positive points.” If that’s an accurate summary, that’s not in conflict with my point.
Can you be more specific about what part of “socially penalize people for being rude in their interactions” would tune down their “cognitive spikiness”? This seems like a false dichotomy, but I’m open to hearing about costs of my proposal I was unaware of.
So if some boss often drove his employees to tears, as long as he was pretty insightful, you don’t think that the employees should be able to know before taking the job? Surely that’s not your position. But then what is?
I wanted to add a perspective to the conversation that I didn’t see mentioned, moreso than advocating a very thought out position. I have conflicting intuitions, and the territory seems messy!
On the one hand, it does seem to me like there should be some kind of “heads up about intensity”. It’s real bad to create hidden slippery slopes along the intensity scale. It’s real bad to first make people dependent on you (by, say, paying most of their salary in yet-to-be-vested equity, making them work long enough that they can’t explore external opportunities and maintain outside friends, …) and then shifting into a potentially abusive stance (heavily frame controlling, demoralising, etc). It is when these kinds of pressures are applied that I think things move into unacceptable territory. (And my suggested community response would probably be something like “Sandbox culprit in ways where they’re able to remain highly productive while doing less damage, give people accurate indications about their style (conveying this might actually fall on someone else than the culprit to do—that division of labor might just be our only way to get all the good stuff here!), and avoid giving people inaccurate impressions or being a wide-eyed feeder school.”
For comparison, when I imagine pursuing a career in investment banking, it seems like I’d be opting into a shark tank. I’m just kind of accepting there’ll be some real abusive folks around, following the $$$, and I’ll be betting on my ability to navigate that without losing myself in the process. Being part of a healthy community means somehow having people around me who can help me see these things. I do think there are some young undergrads who naively will believe the faceless Goldman ads. I feel like Taleb would have a word for them—the “sucker” or the “Intellectual Yet Idiot”. They’ll get hurt, and this is bad, and the recruiting ads that led them into this are immoral.
(From that perspective, I’m pretty into my straw version of military ads, which is more like “You’ll have the worst time of your life and be tested to your limits. You’re too weak for this. But you’ll gain glory. Sign up here.”)
On the other hand, I also have the intuition that requesting of individual researchers that they signpost and warn about their unusual communication style seems to be locating the onus of this in the wrong location… and I kind of just don’t expect it to work, empirically? I feel like the getting-a-job-at-MIRI pipeline should somehow make it clear to people what level of f*ckery is about to happen to them, insofar as it is. I currently don’t know whose responsibility I think that is (and I’m shipping this comment in a confused state, rather than not shipping it at all).
In my opinion, this is a responsibility of the person who made the decision that Nate works for their organization. They should either do it, or delegate it to someone and verify that it is done.
Seconding this. A very large part, perhaps the majority, of the problem with everything that’s been described in this discussion seems to me to boil down to “poor management”. Totally ignoring that and instead focusing either on Nate or on any of the “complainants” here seems just wildly misguided.
My take is that they (those who make such decisions of who runs what) are pretty well-informed about these issues well before they escalate to the point that complaints bubble up into posts / threads like these.
I would have liked this whole matter to have unfolded differently. I don’t think this is merely a sub-optimal way for these kinds of issues to be handled, I think this is a negative one.
I have a number of ideological differences with Nate’s MIRI and Nate himself that I can actually point to and articulate, and those disagreements could be managed in a way that actually resolve those differences satisfactorily. Nate’s MIRI—to me—seemed to be one of the most ideologically conformist iterations of the organization observed thus far.
Furthermore, I dislike that we’ve converged on the conclusion that Nate is a bad communicator, or that he has issues with his personality, or—even worse—that it was merely the lack of social norms imposed on someone with his level of authority that allowed him to behave in ways that don’t jive with many people (implying that literally anyone with such authority would behave in a similar way, without the imposition of more punitive and restrictive norms).
Potentially controversial take: I don’t think Nate is a bad communicator. I think Nate is incorrect about important things, and that incorrect ideas tend to appear to be communicated badly, which accounts for perceptions that he is a bad communicator (and perhaps also accounts for observations that he seemed frustrated and-or distressed while trying to argue for certain things). Whenever I’ve seen him communicate sensible ideas, it seems communicated pretty well to me.
I feel that this position is in fact more respectful to Nate himself.
If we react on the basis of Nate’s leadership style being bad, his communication being bad, or him having a brusque personality, then he’s just going to be quietly replaced by someone who will also run the organization in a similar (mostly ideologically conformist) way. It will be assumed (or rather, asserted) that all organizational issues experienced under his tenure were due to his personal foibles and not due to its various intellectual positions, policies, and strategic postures (e.g. secrecy), all of which are decided upon by other people including Nate, but executed upon by Nate! This is why I call this a negative outcome.
By the way: Whenever I see it said that an idea was “communicated badly” or alternatively that it is more complicated and nuanced than the person ostensibly not-understanding it thinks it should be, I take that as Bayesian evidence of ideological conformity. Given that this is apparently a factor that is being argued for, I have to take it as evidence of that.
It is possible to be a good communicator in some situations (e.g. when you write a blog post) and a bad communicator in other situations (e.g. when someone randomly interrupts you when you were worried about something else).
For example, when I talk, I am much less coherent, and my English sucks.
If I remember the details correctly (sorry, I am not going to read the entire thread again), this seems like a mistake that could be avoided in the future. -- Someone tried to make Nate happy by telling Kurt to do something for him; Nate didn’t ask for any help, but when an attempt was made regardless, he got angry at Kurt because he perceived the help as unreliable, worse than nothing. Kurt was hurt, because this wasn’t his idea in the first place, and he tried to communicate a problem with his task, unsuccessfully. -- I think a possible lesson is to just leave Nate alone, unless he explicitly asks for help, and even then think twice whether you chose the right person for the job. And maybe have someone managing your employees, whom they can ask for advice, if needed.
(Yes, I would prefer if Nate just magically stopped being angry at people who are trying to help, even if he is not satisfied with the outcome. But it is not wise to rely on magic to happen.)
More meta, when people have a bad experience with Nate (or anyone else), don’t ignore that fact. Stop and think about the situation.
If people felt hurt interacting with me, I would want to know it, get some advice how to prevent this outcome, and if the advice doesn’t feel actionable then at least how to avoid such people and/or situations. It doesn’t necessarily mean that someone is a bad person, sometimes people just rub each other the wrong way, but in such case there should be an option to avoid each other.
I hereby push back against the (implicit) narrative that I find the standard community norms costly, or that my communication protocols are “alternative”.
My model is closer to: the world is a big place these days, different people run on different conversation norms. The conversation difficulties look, to me, symmetric, with each party violating norms that the other considers basic, and failing to demonstrate virtues that the other considers table-stakes.
(To be clear, I consider myself to bear an asymmetric burden of responsibility for the conversatiosn going well, according to my seniority, which is why I issue apologies instead of critiques when things go off the rails.)
Separately but relatedly: I think the failure-mode I had with Vivek & co was rather different than the failure-mode I had with you. In short: in your case, I think the issue was rooted in a conversational dynamic that caused me frustration, whereas in Vivek & co’s case, I think the issue was rooted in a conversational dynamic that caused me despair.
Which is not to say that the issues are wholly independent; my guess is that the common-cause is something like “some people take a lot of damage from having conversations with someone who despairs of the conversation”.
Tying this back: my current model of the situation is not that I’m violating community norms about how to have a conversation while visibly hopeless, but am rather in uncharted territory by trying to have those conversations at all.
(For instance: standard academia norms as I understand them are to lie to yourself and/or others about how much hope you have in something, and/or swallow enough of the modesty-pill that you start seeing hope in places I would not, so as to sidestep the issue altogether. Which I’m not personally up for.)
([tone: joking but with a fragment of truth] …I guess that the other norm in academia when academics are hopless about others’ research is “have feuds”, which… well we seem to be doing a fine job by comparison to the standard norms, here!)
Where, to be clear, I already mostly avoid conversations where I’m hopeless! I’m mostly a hermit! The obvious fix of “speak to fewer people” is already being applied!
And beyond that, I’m putting in rather a lot of work (with things like my communication handbook) to making my own norms clearer, and I follow what I think are good meta-norms of being very open to trying other people’s alternative conversational formats.
I’m happy to debate what the local norms should be, and to acknowledge my own conversational mistakes (of which I have made plenty), but I sure don’t buy a narrative that I’m in violation of the local norms.
(But perhaps I will if everyone in the comments shouts me down! Local norms are precisely the sort of thing that I can learn about by everyone shouting me down about this!)
This is preposterous.
I’m not going to discuss specific norms. Discussing norms with Nate leads to an explosion of conversational complexity.[1] In my opinion, such discussion can sound really nice and reasonable, until you remember that you just wanted him to e.g. not insult your reasoning skills and instead engage with your object-level claims… but somehow your simple request turns into a complicated and painful negotiation. You never thought you’d have to explain “being nice.”
Then—in my experience—you give up trying to negotiate anything from him and just accept that he gets to follow whatever “norms” he wants.
So, in order to evaluate whether Nate is “following social norms”, let’s not think about the norms themselves. I’m instead going to share some more of the interactions Nate has had:
“Flipping out” at Kurt Brown because there wasn’t enough sourdough bread.
Storming out of the room because Kurt had a question about pumping Nate’s tires.
A chat with me, where Nate describes himself as “visibly flustered, visibly frustrated, had a raised voice, and was being mean in various of his replies.”
An employee reporting fear and dread at the prospect of meeting with Nate,
People crying while talking with Nate about research,
ETA: Peter Barnett writes “I think it’s kind of hard to see how bad/annoying/sad this is until you’re in it.”
(There are people who want to speak out, but have not yet. I am not at liberty to detail their cases myself.)
EDIT: Nate did not warn most of these people beforehand. It seems to me that Nate is still balking at requests to warn people.
See the second half of Nate’s response to Akash, starting with “These sound more to me like”.
Regarding my own experience: I would destroy 10% of my liquidity to erase from my life that conversation and its emotional effects.
I don’t think it’s reasonable to expect Nate to have predicted that I in particular would be hurt so much. But in fact, being unexpectedly and nonconsensually mean and aggressive and hurtful has heavy negative tails.
And so statistically, this level of harm is totally predictable, both on priors and based off of past experience which I know Nate has.
This seems like a fairly extreme statement, so I was about to upvote due to the courage required to post it publicly and stand behind it. But then I stopped and thought about the long term effects and it’s probably best not to encourage this.
As ideally, you, along with the vast majority of potential readers, should become less emotionally reactive over time to any real or perceived insults, slights, etc...
If it’s the heat of the moment talking, that’s fine, but letting thoughts of payback, revenge, etc., linger on for days afterwards likely will not lead to any positive outcome.
I have had these thoughts many times. I would berate myself for letting it get on my nerves so much. It was just an hour-and-a-half chat. But I don’t think it’s a matter of “letting” thoughts occur, or not. Certain situations are damaging to certain people, and this situation isn’t a matter of whether people are encouraged to be damaged or not (I certainly had no expectation of writing about this, back in July–October 2022.)
EDIT: Moving another part elsewhere.
I upvoted for effort because it’s clear you put in quite a bit of effort in writing this comment, but skipped expressing agreement or disagreement.
I had thought of several possible responses, and it is worthy of a substantial response, but since it’s not my role to be the adjudicator or corrector of LW users, I’ll pose you this question:
Consider, is it possible for him to take offence in return and then retaliate via some mean(s)? If that does occur, what’s the range of likely outcomes?
(Note that I moved most of the original comment and plan to put it elsewhere in the thread.)
I don’t follow. I’m not going to behave differently in the face of any possible retaliation, nor do I in fact expect Nate to retaliate in an inappropriate manner. So I’m not worried about this?
It seems weird to single out this specific type of human limitation (compared to perfect-robot instrumental rationality) over the hundreds of others. If someone isn’t in top physical shape or cannot drive cars under difficult circumstances or didn’t renew their glasses and therefore doesn’t see optimally, would you also be reluctant to upvote comments you were otherwise tempted to upvote (where they bravely disclose some limitation) because of this worry about poor incentives? “Ideally,” in a world where there’s infinite time so there are no tradeoffs for spending self-improvement energy, rationalists would all be in shape, have brushed up their driving skills, have their glasses updated, etc. In reality, it’s perfectly fine/rational to deprioritize many things that are “good to have” because other issues are more pressing, more immediately deserving of self-improvement energy. (Not to mention that rationality for its own sake is lame anyway and so many of us actually want to do object-level work towards a better future.) What to best focus on with self-improvement energy will differ a lot from person to person, not only because people have different strengths and weaknesses, but also because they operate in different environments. (E.g., in some environments, one has to deal with rude people all the time, whereas in others, this may be a rare occurrence.) For all these reasons, it seems weirdly patronizing to try to shape other people’s prioritization for investing self-improvement energy. This isn’t to say that this site/community shouldn’t have norms and corresponding virtues and vices. Since LW is about truth-seeking, it makes sense to promote virtues directly related to truth-seeking, e.g., by downvoting comments that exhibit poor epistemic practices. However, my point is that even though it might be tempting to discourage not just poor epistemic rationality but also poor instrumental rationality, these two work very differently, especially as far as optimal incentive-setting is concerned. Epistemic rationality is an ideal we can more easily enforce and get closer towards. Instrumental rationality, by contrast, is a giant jungle that people are coming into from all kinds of different directions. “Having unusually distracting emotional reactions to situations xyz” is one example of suboptimal instrumental rationality, but so is “being in poor physical shape,”or “not being able to drive a car,” or “not having your glasses updated,” etc. I don’t think it makes sense for the community to create a hierarchy of “most important facets of instrumental rationality” that’s supposed to apply equally to all kinds of people. (Instead, I think it makes more sense to reward meta-skills of instrumental rationality, such as “try to figure out what your biggest problems are and really prioritize working on them.) (If we want to pass direct judgment on someone’s prioritization of self-improvement energy, we need to know their exact situation and goals and the limitations they have, how good they are at learning various things, etc.) Not to mention the unwelcoming effects when people get judged for limitations of instrumental rationality that the community for some reason perceives to be particularly bad. Such things are always more personal (and therefore more unfair) than judging someone for having made a clear error of reasoning (epistemic rationality).
(I say all of this as though it’s indeed “very uncommon” to feel strongly hurt and lastingly affected by particularly harsh criticism. I don’t even necessarily think that this is the case: If the criticism comes from a person with high standing in a community one cares about, it seems like a potentially quite common reaction?)
This is relevant context for my strong reaction. I used to admire Nate, and so I was particularly upset when he treated me disrespectfully. (The experience wasn’t so much “criticism” as “aggression and meanness”, though.)
FWIW, I also reject the framing that this situation is reasonably understood as an issue with my own instrumental rationality.
Going back to the broader point about incentives, it’s not very rewarding to publicly share a distressing experience and thereby allow thousands of internet strangers to judge my fortitude, and complain if they think it lacking. I’m not walking away from this experience feeling lavished and reinforced for having experienced an emotional reaction.
Furthermore, the reason I spoke up was mostly not to litigate my own experience. It’s because I’ve spent months witnessing my friends take unexpected damage from a powerful individual who appears to have faced basically no consequences for his behavior.
This is a minor error but I feel the need to correct it for future readers, as it’s in the first sentence. There are infinitely many ‘specific types’ of human limitations, or at least an uncountable quantity , depending on the reader’s preferred epistemology.
The rest of your thesis is interesting though a bit difficult to parse. Could you isolate a few of the key points and present them in a list?
I wasn’t the one who downvoted your reply (seems fair to ask for clarifications), but I don’t want to spend much more time on this and writing summaries isn’t my strength. Here’s a crude attempt at saying the same thing in fewer and different words:
IMO, there’s nothing particularly “antithetical to LW aims/LW culture” (edit: “antithetical to LW aims/LW culture” is not a direct quote by anyone; but it’s my summary interpretation of why you might be concerned about bad incentives in this case) about neuroticism-related “shortcomings.” “Shortcomings” compared to a robotic ideal of perfect instrumental rationality. By “neuroticism-related “shortcomings”″, I mean things like having triggers or being unusually affected by harsh criticism. It’s therefore weird and a bit unfair to single out such neuroticism-related “shortcomings” over things like “being in bad shape” or “not being good at common life skills like driving a car.” (I’m guessing that you wouldn’t be similarly concerned about setting bad incentives if someone admitted that they were bad at driving cars or weren’t in the best shape.) I’m only guessing here, but I wonder about rationalist signalling cascades about the virtues of rationality, where it gets rewarded to be particularly critical about things that least correspond to the image of what an ideally rational robot would be like. However, in reality, applied rationality isn’t about getting close to some ideal image. Instead, it’s about making the best out of what you have, taking the best next move step-by-step for your specific situation, always prioritizing what actually gets you to your goals rather than prioritizing “how do I look as though I’m very rational.”
Not to mention that high emotionality confers advantages in many situations and isn’t just an all-out negative. (See also TurnTrout’s comment about rejecting the framing that this is an issue of his instrumental rationality being at fault.)
I don’t mind the occasional downvote or negative karma, it even has some positive benefits, such as being a useful signalling function. As it’s decent evidence I haven’t tailored my comments for popularity or platitudes.
In regards to your points, I’ll only try to respond to them one at a time, since this is already pretty far down the comment chain.
Who suggested that there was a relation between being “antithetical to LW aims/LW culture” and “neuroticism-related “shortcomings”″?
i.e. Is it supposed to be my idea, TurnTrout’s, your’s, a general sentiment, something from the collective unconscious, etc.?
I made an edit to my above comment to address your question; it’s probably confusing that I used quotation marks for something that wasn’t a direct quote by anyone.
I appreciate the edit though can you clarify why you put so many quotes in when they are your own thoughts?
Is it just an idiosyncratic writing style or is it also meant to convey some emotion, context, direction, etc.?
But to clarify, this is not the reason why I ‘might be concerned about bad incentives in this case’, if you were wondering.
Not Lukas, but I also sometimes use quotes:
as a kind of semantic brackets; I think the official way to do this is to write-the-words-connected-by-hyphens, but that just seems hard to read;
to remove a possible connotation, i.e. to signal that I am using the word not exactly as most people would probably use it in a similar situation;
or as a combination of both, something like: I am using these words to express an idea, but these are probably not the right words, but I can’t find any better, so please do not take this part literally and don’t start nitpicking (don’t assume that I used a specific word because I wanted to hint at something specific).
For example, as I understand it,
Means: things that technically are shortcomings (because they deviate from some ideal), but also a reasonable person wouldn’t call them so (because it is a normal human behavior, and I would actually be very suspicious about anyone who claimed to not have any of them), so the word is kinda correct but also kinda incorrect. But it is a way to express what I mean.
Sounds like I misinterpreted the motivation behind your original comment!
I ran out of energy to continue this thread/conversation, but feel free to clarify what you meant for others (if you think it isn’t already clear enough for most readers).
(For completeness, I want to note that I’ve talked with a range of former/current MIRI employees, and a non-trivial fraction did have basically fine interactions with Nate.)
More detail here seems like it could be good. What form did the insult take? Other relevant context?
According to my notes and emails, Nate repeatedly said things like “I have not yet ruled out [uncharitable hypothesis about how TurnTrout is reasoning]” in order to—according to him—accomplish his conversational objectives / because his “polite” statements were apparently not getting his point across. I don’t remember what specific uncharitable things he said (the chat was on 7/19/22).
I might come back and add more context later.
Thank you.
Huh, I initially found myself surprised that Nate thinks he’s adhering to community norms. I wonder if part of what’s going on here is that “community norms” is a pretty vague phrase that people can interpret differently.
Epistemic status: Speculative. I haven’t had many interactions with Nate, so I’m mostly going off of what I’ve heard from others + general vibes.
Some specific norms that I imagine Nate is adhering to (or exceeding expectations in):
Honesty
Meta-honesty
Trying to offer concrete models and predictions
Being (internally) open to acknowledging and recognizing mistakes, saying oops, etc.
Some specific norms that I think Nate might not be adhering to:
Engaging with people in ways such that they often feel heard/seen/understood
Engaging with people in ways such that they rarely feel dismissed/disrespected
Something fuzzy that lots of people would call “kindness” or “typical levels of warmth”
I’m guessing that some people think that social norms dictate something like “you are supposed to be kind and civil and avoid making people unnecessarily sad/insecure/defensive.” I wonder if Nate (a) believes that these are community norms and thinks he’s following them or (b) just doesn’t think these are community norms in the first place.
I think this explains some of the effect, but not all of it. In academia, for instance, I think there are plenty of conversations in which two researchers (a) disagree a ton, (b) think the other person’s work is hopeless or confused in deep ways, (c) honestly express the nature of their disagreement, but (d) do so in a way where people generally feel respected/valued when talking to them.
Like, it’s certainly easier to make people feel heard/seen if you agree with them a bunch and say their ideas are awesome, but of course that would be dishonest [for Nate].
So I could see a world where Nate is like “Darn, the reason people seem to find communicating with me to be difficult is that I’m just presenting them with the harsh truths, and it is indeed hard for people to hear harsh truths.”
But I think some people possess the skill of “being able to communicate harsh truths accurately in ways where people still find the interaction kind, graceful, respectful, and constructive.” And my understanding is that’s what people like TurnTrout are wishing for.
This is not a reasonable norm. In some circumstances (including, it sounds like, some of the conversations under discussion) meeting this standard would require a large amount of additional effort, not related to the ostensible reason for talking in the first place.
Again, a pretty unreasonable norm. For some topics, such as “is what you’re doing actually making progress towards that thing you’ve arranged your life (including social context) around making progress on?”, it’s very easy for people to feel this way, even if they are being told true, useful, relevant things.
Ditto, though significantly less strongly; I do think there’s ways to do this that stay honest and on-mission without too much tradeoff.
I think it’s not a reasonable norm to make sure your interlocutors never e.g. feel dismissed/disrespected, but it is reasonable to take some measures to avoid having someone consistently feel dismissed/disrespected if you spend over 200 hours talking with their team and loosely mentoring them (which to be clear Nate did, it’s just difficult in his position and so was only mildly successful).
I’m not sure kindness/warmth should even be a norm because it’s pretty difficult to define.
The details matter here; I don’t feel I can guess from what you’ve said whether we’d agree or not.
For example:
Tam: says some idea about alignment
Newt: says some particular flaw ”...and this is an instance of a general problem, which you’ll have to address if you want to make progress...” gestures a bit at the general problem
Tam: makes a tweak to the proposal that locally addresses the particular flaw
Newt: “This still doesn’t address the problem.”
Tam: “But it seems to solve the concrete problem, at least as you stated it. It’s not obvious to me that there’s a general problem here; if we can solve instances of it case-by-case, that seems like a lot of progress.”
Newt: “Look, we could play this game for some more rounds, where you add more gears and boxes to make it harder to see that there’s a problem that isn’t being addressed at all, and maybe after a few rounds you’ll get the point. But can we just skip ahead to you generalizing to the class of problem, or at least trying to do that on your own?”
Tam: feels dismissed/disrespected
I think Newt could have been more graceful and more helpful, e.g. explicitly stating that he’s had a history of conversations like this, and setting boundaries about how much effort he feels exciting about putting in, and using body language that is non-conflictual… But even if he doesn’t do that, I don’t really think he’s violating a norm here. And depending on context this sort of behavior might be about as well as Newt can do for now.
You can choose to ignore all these “unreasonable norms”, but they still have consequences. Such as people thinking you are an asshole. Or leaving the organization because of you. It is easy to underestimate these costs, because most of the time people won’t tell you (or they will, but you will ignore them and quickly forget).
This is a cost that people working with Nate should not ignore, even if Nate does.
I see three options:
try making Nate change—this may not be possible, but I think it’s worth trying;
isolate Nate from… well, everyone else, except for volunteers who were explicitly warned;
hire a separate person whose full time job will be to make Nate happy.
Anything else, I am afraid, will mean paying the costs and most likely being in denial about them.
I see at least two other options (which, ideally, should be used in tandem):
don’t hire people who are so terribly sensitive to above-average blutness
hire managers who will take care of ops/personnel problems more effectively, thus reducing the necessity for researchers to navigate interpersonal situations that arise from such problems
If I translate it mentally to “don’t hire people from the bottom 99% of thick skin”, I actually agree. Though they may be difficult to find, especially in combination with other requirements.
Are you available for the job? ;-)
Do you really think it’d take 99th percentile skin-thickness to deal with this sort of thing without having some sort of emotional breakdown? This seems to me to be an extraordinary claim.
While I probably qualify in this regard, I don’t think that I have any other relevant qualifications.
My experience is that people who I think of as having at least 90th percentile (and probably 99th if I think about it harder) thick-skin have been brought to tears from an intense conversation with Nate.
My guess is that this wouldn’t happen for a lot of possible employees from the broader economy, and this isn’t because they’ve got thicker skin, but it’s because they’re not very emotionally invested in the organization’s work, and generally don’t bring themselves to their work enough to risk this level of emotion/hurt.
This is a truly extraordinary claim! I don’t know what evidence I’d need to see in order to believe it, but whatever that evidence is, I sure haven’t seen it yet.
This just can’t be right. I’ve met a decent number of people who are very invested in their work and the mission of whatever organization they’re part of, and I can’t imagine them being brought to tears by “an intense conversation” with one of their co-workers (nor have I heard of such a thing happening to the people I have in mind).
Something else is going on here, it seems to me; and the most obvious candidate for what that “something else” might be is simply that your view of what the distribution of “thick-skinned-ness” is like, is very mis-calibrated.
To me the obvious candidate is that people are orienting around Nate in particular in an especially weird way.
(Don’t know why some folks have downvoted the above comment, seems like a totally normal epistemic state for Person A not to believe what Person B believes about something after simply learning that Person B believes it, and to think Person B is likely miscalibrated. I have strong upvoted the comment back to clearly positive.)
My model says that this requires them to still be hopeful about local communication progress, and happens when they disagree but already share a lot of frames and concepts and background knowledge. I, at least, find it much harder when I don’t expect the communciation attempt to make progress, or have positive effect.
(“Then why have the conversation at all?” I mostly don’t! But sometimes I mispredict how much hope I’ll have, or try out some new idea that doesn’t work, or get badgered into it.)
These sound more to me like personality traits (that members of the local culture generally consider virtuous) than communication norms.
On my model, communciation norms are much lover-level than this. Basics of rationalist discourse seem closer; archaic politeness norms (“always refuse food thrice before accepting”) are an example of even lower-level stuff.
My model, speaking roughly and summarizing a bunch, says that the lowest-level stuff (atop a background of liberal-ish internet culture and basic rationalist discourse) isn’t pinned down on account of cultural diversity, so we substitute with meta-norms, which (as best I understand them) include things like “if your convo-partner requests a particular conversation-style, either try it out or voice objections or suggest alternatives” and “if things aren’t working, retreat to a protected meta discussion and build a shared understanding of the issue and cooperatively address it”.
I acknowledge that this can be pretty difficult to do on the fly, especially if emotions are riding high. (And I think we have cultural diversity around whether emotions are ever supposed to ride high, and if so, under what circumstances.) On my model of local norms, this sort of thing gets filed under “yep, communicating in the modern world can be rocky; if something goes wrong then you go meta and try to figure out the causes and do something differently next time”. (Which often doesn’t work! In which case you iterate, while also shifting your conversational attention elsewhere.)
To be clear, I buy a claim of the form “gosh, you (Nate) seem to run on a relatively rarer native emotional protocol, for this neck of the woods”. My model is that local norms are sufficiently flexible to continue “and we resolve that by experimentation and occasional meta”.
And for the record, I’m pretty happy to litigate specific interactions. When it comes to low-level norms, I think there are a bunch of conversational moves that others think are benign that I see as jabs (and which I often endorse jabbing back against, depending on the ongoing conversation style), and a bunch of conversational moves that I see as benign that others take as jabs, and I’m both (a) happy to explicate the things that felt to me like jabs; (b) happy to learn what other people took as jabs; and (c) happy to try alternative communication styles where we’re jabbing each other less. Where this openness-to-meta-and-trying-alternative-things seems like the key local meta-norm, at least in my understanding of local culture.
It seems to me that in theory it should be possible to have very unusual norms and make it work, but that in practice you and your organization horribly underestimate how difficult it is to communicate such things clearly (more than once, because people forget or don’t realize the full implications at the first time). You assume that the local norms were made perfectly clear, but they were not (expecting short inferential distances, double illusion of transparency, etc.).
Did you expect KurtB to have this kind of reaction, to post this kind of comment, and to get upvoted? If the answer is no, it means your model is wrong somewhere.
(If the answer is yes, maybe you should print that comment, and give a copy to all new employees. That might dramatically reduce a possibility of misunderstanding.)
My original comment is not talking about communication norms. It’s talking about “social norms” and “communication protocols” within those norms. I mentioned “basic respectfulness and professionalism.”
This is a thing, but I’m guessing that what you have in mind involves a lot more than you’re crediting of not actually trying for the crux of the conversation. As just one example, you can be “more respectful” by making fewer “sweeping claims” such as “you are making such and such error in reasoning throughout this discussion / topic / whatever”. But that’s a pretty important thing to be able to say, if you’re trying to get to real cruxes and address despair and so on.
Kinda. I’m advocating less for the skill of “be graceful and respectful and constructive” and instead looking at the lower bar of “don’t be overtly rude and aggressive without consent; employ (something within 2 standard deviations of) standard professional courtesy; else social consequences.” I want to be clear that I’m not wishing for some kind of subtle mastery, here.
Nate, I am skeptical.
As best I can fathom, you put in very little work to proactively warn new hires about the emotional damage which your employees often experience. I’ve talked to a range of people who have had professional interactions with you, both recently and further back. Only one of the recent cases reported that you warned them before they started working with you.
In particular, talking to the hires themselves, I have detected no evidence that you have proactively warned most of the hires[1] you’ve started working with since July 2022, which is when:
I told you that your anger and ranting imposed unexpected and large costs on me,
And you responded with something like “Sorry, I’ll make sure to tell people I have research conversations with—instead of just my formal collaborators. Obvious in hindsight.”
And yet you apparently repeatedly did not warn most of your onboarded collaborators.
EDIT: The original version of this comment claimed “None were warned.” This was accurate reporting at the time of my comment. However, I now believe that Nate did in fact proactively warn Vivek, at least a bit and to some extent. I am overall still worried, as I know of several specific cases which lacked sufficient warning, and some of them paid surprisingly high costs because of it.
Vivek did not recall any warning but thought it possible that you had verbally mentioned some cons, which he forgot about. However, now that Nate jogged his memory a bit, he thinks he probably did receive a warning.
I personally wouldn’t count a forgettable set of remarks as “sufficient warning”, given the track record here. It sure seems to me like it should have been memorable.
On the facts: I’m pretty sure I took Vivek aside and gave a big list of reasons why I thought working with me might suck, and listed that there are cases where I get real frustrated as one of them. (Not sure whether you count him as “recent”.)
My recollection is that he probed a little and was like “I’m not too worried about that” and didn’t probe further. My recollection is also that he was correct in this; the issues I had working with Vivek’s team were not based in the same failure mode I had with you; I don’t recall instances of me getting frustrated and bulldozey (though I suppose I could have forgotten them).
(Perhaps that’s an important point? I could imagine being significantly more worried about my behavior here if you thought that most of my convos with Vivek’s team were like most of my convos with you. I think if an onlooker was describing my convo with you they’d be like “Nate was visibly flustered, visibly frustrated, had a raised voice, and was being mean in various of his replies.” I think if an onlooker was describing my convos with Vivek’s team they’d be like “he seemed sad and pained, was talking quietly and as if choosing the right words was a struggle, and would often talk about seemingly-unrelated subjects or talk in annoying parables, while giving off a sense that he didn’t really expect any of this to work”. I think that both can suck! And both are related by a common root of “Nate conversed while having strong emotions”. But, on the object level, I think I was in fact avoiding the errors I made in conversation with you, in conversation with them.)
As to the issue of not passing on my “working with Nate can suck” notes, I think there are a handful of things going on here, including the context here and, more relevantly, the fact that sharing notes just didn’t seem to do all that much in practice.
I could say more about that; the short version is that I think “have the conversation while they’re standing, and I’m lying on the floor and wearing a funny hat” seems to work empirically better, and...
hmm, I think part of the issue here is that I was thinking like “sharing warnings and notes is a hypothesis, to test among other hypotheses like lying on the floor and wearing a funny hat; I’ll try various hypotheses out and keep doing what seems to work”, whereas (I suspect) you’re more like “regardless of what makes the conversations go visibly better, you are obligated to issue warnings, as is an important part of emotionally-bracing your conversation partners; this is socially important if it doesn’t seem to change the conversation outcomes”.
I think I’d be more compelled by this argument if I was having ongoing issues with bulldozing (in the sense of the convo we had), as opposed to my current issue where some people report distress when I talk with them while having emotions like despair/hoplessness.
I think I’d also be more compelled by this argument if I was more sold on warnings being the sort of thing that works in practice.
Like… (to take a recent example) if I’m walking by a whiteboard in rosegarden inn, and two people are like “hey Nate can you weigh in on this object-level question”, I don’t… really believe that saying “first, be warned that talking techincal things with me can leave you exposed to unshielded negative-valence emotions (frustration, despair, …), which some people find pretty crappy; do you still want me to weigh in?” actually does much. I am skeptical that people say “nope” to that in practice.
I suppose that perhaps what it does is make people feel better if, in fact, it happens? And maybe I’ll try it a bit and see? But I don’t want to sound like I’m promising to do such a thing reliably even as it starts to feel useless to me, as opposed to experimenting and gravitating towards things that seem to work better like “offer to lie on the floor while wearing a funny hat if I notice things getting heated”.
I’ve been asked to clarify a point of fact, so I’ll do so here:
This does ring a bell, and my brain is weakly telling me it did happen on a walk with Nate, but it’s so fuzzy that I can’t tell if it’s a real memory or not. A confounder here is that I’ve probably also had the conversational route “MIRI burnout is a thing, yikes” → “I’m not too worried, I’m a robust and upbeat person” multiple times with people other than Nate.
In private correspondence, Nate seems to remember some actual details, and I trust that he is accurately reporting his beliefs. So I’d mostly defer to him on questions of fact here.
I’m pretty sure I’m the person mentioned in TurnTrout’s footnote. I confirm that, at the time he asked me, I had no recollection of being “warned” by Nate but thought it very plausible that I’d forgotten.
This is a slight positive update for me. I maintain my overall worry and critique: chats which are forgettable do not constitute sufficient warning.
Insofar as non-Nate MIRI personnel thoroughly warned Vivek, that is another slight positive update, since this warning should reliably be encountered by potential hires. If Vivek was independently warned via random social connections not possessed by everyone,[1] then that’s a slight negative update.
For example, Thomas Kwa learned about Nate’s comm doc by randomly talking with a close friend of Nate’s, and mentioning comm difficulties.
I think there are several critical issues with your behavior, but I think the most urgent is that people often don’t know what they’re getting into. People have a right to make informed decisions and to not have large, unexpected costs shunted onto them.
It’s true that no one has to talk with you. But it’s often not true that people know what they’re getting into. I spoke out publicly because I encountered a pattern, among my friends and colleagues, of people taking large and unexpected emotional damage from interacting with you.
If our July interaction had been an isolated incident, I still would have been quite upset with you, but I would not have been outraged.
If the pattern I encountered were more like “a bunch of people report high costs imposed by Nate, but basically in the ways they expected”, I’d be somewhat less outraged.[1] If people can accurately predict the costs and make informed decisions, then people who don’t mind (like Vivek or Jeremy) can reap the benefits of interacting with you, and the people who would be particularly hurt can avoid you.
If your warnings are not preventing this pattern of unexpected hurt, then you need to do better. You need to inform people to the point that they know what distribution they’re sampling from. If people know, I’m confident that they will start saying “no.” I probably would have said “no thanks” (or at least ducked out sooner and taken less damage), and Kurt would have said “no” as well.
If you don’t inform people to a sufficient extent, the community should (and, I think, will) hold you accountable for the unexpected costs you impose on others.
I would still be disturbed and uneasy for the reasons Jacob Steinhardt mentioned, including “In the face of real consequences, I think that Nate would better regulate his emotions and impose far fewer costs on people he interacts with.”
(I don’t know who strong disagree-voted the parent comment, but I’m interested in hearing what the disagreement is. I currently think the comment is straightforwardly correct and important.)
The 9-karma disagree-vote is mine. (Surprise!) I thought about writing a comment, and then thought, “Nah, I don’t feel like getting involved with this one; I’ll just leave a quick disagree-vote”, but if you’re actively soliciting, I’ll write the comment.
I’m wary of the consequences of trying to institute social norms to protect people from subjective emotional damage, because I think “the cure is worse than the disease.” I’d rather develop a thick skin and take responsibility for my own emotions (even though it hurts when some people are mean), because I fear that the alternative is (speaking uncharitably) a dystopia of psychological warfare masquerading as kindness in which people compete to shut down the expression of perspectives they don’t like by motivatedly getting (subjectively sincerely) offended.
Technically, I don’t disagree with “people should know what they’re getting into” being a desirable goal (all other things being equal), but I think it should be applied symmetrically, and it makes sense for me to strong-disagree-vote a comment that I don’t think is applying it symmetrically: it’s not fair if “fighty” people need to to make lengthy disclaimers about how their bluntness might hurt someone’s feelings (which is true), but “cooperative” people don’t need to make lengthy disclaimers about how their tone-policing might silence someone’s perspective (which is also true).
I don’t know Nate very well. There was an incident on Twitter and Less Wrong the other year where I got offended at how glib and smug he was being, despite how wrong he was about the philosophy of dolphins. But in retrospect, I think I was wrong to get offended. (I got downvoted to oblivion, and I deserved it.) I wish I had kept my cool—not because I personally approve of the communication style Nate was using, but because I think it was bad for my soul and the world to let myself get distracted by mere style when I could have shrugged it off and stayed focused on the substance.
You told me you would warn people, and then did not.[1]
Do I have your permission to quote the relevant portion of your email to me?
I warned the immediately-next person.
It sounds to me like you parsed my statement “One obvious takeaway here is that I should give my list of warnings-about-working-with-me to anyone who asks to discuss their alignment ideas with me, rather than just researchers I’m starting a collaboration with.” as me saying something like “I hereby adopt the solemn responsibility of warning people in advance, in all cases”, whereas I was interpreting it as more like “here’s a next thing to try!”.
I agree it would have been better of me to give direct bulldozing-warnings explicitly to Vivek’s hires.
Here is the statement:
I agree that this statement does not explicitly say whether you would make this a one-time change or a permanent one. However, the tone and phrasing—”Obvious in hindsight; sorry for not doing that in your case”—suggested that you had learned from the experience and are likely to apply this lesson going forward. The use of the word “obvious”—twice—indicates to me that you believed that warnings are a clear improvement.
Ultimately, Nate, you wrote it. But I read it, and I don’t really see the “one-time experiment” interpretation. It just doesn’t make sense to me that it was “obvious in hindsight” that you should… adopt this “next thing to try”..?
I did not intend it as a one-time experiment.
In the above, I did not intend “here’s a next thing to try!” to be read like “here’s my next one-time experiment!”, but rather like “here’s a thing to add to my list of plausible ways to avoid this error-mode in the future, as is a virtuous thing to attempt!” (by contrast with “I hereby adopt this as a solemn responsibility”, as I hypothesize you interpreted me instead).
Dumping recollections, on the model that you want more data here:
I intended it as a general thing to try going forward, in a “seems like a sensible thing to do” sort of way (rather than in a “adopting an obligation to ensure it definitely gets done” sort of way).
After sending the email, I visualized people reaching out to me and asking if i wanted to chat about alignment (as you had, and as feels like a reconizable Event in my mind), and visualized being like “sure but FYI if we’re gonna do the alignment chat then maybe read these notes first”, and ran through that in my head a few times, as is my method for adopting such triggers.
I then also wrote down a task to expand my old “flaws list” (which was a collection of handles that I used as a memory-aid for having the “ways this could suck” chat, which I had, to that point, been having only verbally) into a written document, which eventually became the communication handbook (there were other contributing factors to that process also).
An older and different trigger (of “you’re hiring someone to work with directly on alignment”) proceeded to fire when I hired Vivek (if memory serves), and (if memory serves) I went verbally through my flaws list.
Neither the new nor the old triggers fired in the case of Vivek hiring employees, as discussed elsewhere.
Thomas Kwa heard from a friend that I was drafting a handbook (chat logs say this occured on Nov 30); it was still in a form I wasn’t terribly pleased with and so I said the friend could share a redacted version that contained the parts that I was happier with and that felt more relevant.
Around Jan 8, in an unrelated situation, I found myself in a series of conversations where I sent around the handbook and made use of it. I pushed it closer to completion in Jan 8-10 (according to Google doc’s history).
The results of that series of interactions, and of Vivek’s team’s (lack of) use of the handbook caused me to update away from this method being all that helpful. In particular: nobody at any point invoked one of the affordances or asked for one of the alternative conversation modes (though those sorts of things did seem to help when I personally managed to notice building frustration and personally suggest that we switch modes (although lying on the ground—a friend’s suggestion—turned out to work better for others than switching to other conversation modes)). This caused me to downgrade (in my head) the importance of ensuring that people had access to those resources.
I think that at some point around then I shared the fuller guide with Vivek’s team, but I didn’t quickly detemine when from the chat logs. Sometime between Nov 30 and Feb 22, presumably.
It looks from my chat logs like I then finished the draft around Feb 22 (where I have a timestamp from me noting as much to a friend). I probably put it publicly on my website sometime around then (though I couldn’t easily find a timestamp), and shared it with Vivek’s team (if I hadn’t already).
The next two MIRI hires both mentioned to me that they’d read my communication handbook (and I did not anticipate spending a bunch of time with them, nevermind on technical research), so they both didn’t trigger my “warn them” events and (for better or worse) I had them mentally filed away as “has seen the affordances list and the failure modes section”.
I appreciate the detail, thanks. In particular, I had wrongly assumed that the handbook had been written much earlier, such that even Vivek could have been shown it before deciding to work with you. This also makes more sense of your comments that “writing the handbook” was indicative of effort on your part, since our July interaction.
Overall, I retain my very serious concerns, which I will clarify in another comment, but am more in agreement with claims like “Nate has put in effort of some kind since the July chat.”
Noting that at least one of them read the handbook because I warned them and told them to go ask around about interacting with you, to make sure they knew what they were getting into.
Yep! I’ve also just reproduced it here, for convenience:
Surely the problem with “state non-obvious takes in an obvious tone, and decline to elaborate” bulldozing is that it violates the maxim that beliefs should be supported with arguments and evidence? (And the reason for the maxim is that even the smartest human experts aren’t infallible; if not subjected to the rigor of the debate algorithm, they’re going to get things wrong.) It seems misplaced to focus on emotionally bad experiences and punctured excitement.
That’s relevant, but I’m largely not discussing group epistemics. I’m discussing the community impact of social norms. That impact is measured in human well-being, productivity, and happiness, as well as the height of the sanity waterline. Concretely—if I treat my colleagues in a rude and angry manner, that imposes costs on them. In that situation, whether or not I’m making correct verbal claims, that’s generally not a good community to be a part of, and it’s not a good way to treat people.
Emotionally bad experiences are an extremely relevant quantity to discuss.
(I don’t expect to engage further due to our past unfruitful interactions on similar topics.)
EDIT: Also, clarification that the “bulldozing” incident did not primarily consist of “state non-obvious takes in an obvious tone, and decline to elaborate.”
What did it consist of? Have you described it somewhere?
Nate did this.
By my somewhat idiosyncratic views on respectful communication, Nate was roughly as respectful as Thomas Kwa.
I do seem to be unusually emotionally compatible with Nate’s style of communication though.
I agree with a couple of caveats. Even though Nate did try pretty hard and even send us a guide on communicating with him, he’s still difficult (for me) to work with even after reading the guide. I think I’m more capable of minimizing unintentional emotional damage than Nate is just due to our demeanors.
Edit: I misremembered; Nate didn’t send us the guide proactively, someone who knew him sent it to us after we asked for tips on communicating with Nate.
Woah! That’s like 10x more effort than I expect >90% of difficult-to-communicate-with people will go through.
Kudos to Nate for that.
For convenience: Nate-culture communication handbook
I found that an endearing document to read. A lot of clear self-reflection communicated explicitly, describing both endorsed and unendorsed self-properties, in an effort to help others communicate and interface.
The document is interesting, but how well does it describe Nate’s actual behavior? Can you find the parts that correspond to this:
As I wrote elsewhere:
The 12 section headings under “Failure modes” are: Confidence, Frustration, Lashing out, Condescension, Nonrespect, Dismissal, Disdain, Disbelieving you, Harshness, Tunnel vision, Retaliation, and Stoneface. The elements in your quoted section appear to me to come up here (e.g. Lashing out, Tunnel vision, Dismissal, etc).
I don’t say it’s a successful document. My first guess is that a more successful version of the doc (edit: or at least a doc that successfully conveys what to expect when interacting with the author) would be shorter and focus on giving a more grounded sense of “here are concrete instances of the peak good and bad interactions I’ve had with people, and some sense of the modal interaction type”. It might include content like “I’ve talked to N people who’ve said they’ve had life-changingly positive interactions with me, and M people who’ve said as a result they were really substantively hurt by interacting with me and wish to indefinitely avoid me, here’s some properties of the interactions they described, also here’s a survey on general properties of how people find me to be in conversation, split out between friends and colleagues”.
However I think that the sort of doc I’m describing is ~unheard of in any setting and way more effort than ~anyone I’m aware of has put into this sort of widespread expectation setting with lots of colleagues and collaborators (I’d say the same for the linked doc that Nate drafted), and people typically do not have much obligation to let people know about bad interactions. Heck, in many countries people can get criminal records expunged so that they don’t have to inform their future employers about them, which is worlds apart from handing someone a doc listing times when people they’ve talked to have felt burned by the interaction, which reads to me like a standard being demanded elsethread.
There are things that I really like about the document, but I feel like I’d need to know more about its reason for being created to say whether this deserves kudos.
It seems plausible that the story went something like this: “Nate had so much social standing that he was allowed/enabled to do what most ‘difficult to interact with’ people couldn’t, namely to continue in their mannerisms without making large changes, and still not suffer from a reduction of social standing. Partly to make this solution palatable to others and to proactively address future PR risks and instances of making people sad (since everyone already expected/was planning for more such instances to come up), Nate wrote this document.”
If an org is going to have this sort of approach to its most senior researcher, it’s still better to do it with a document of this nature than without.
But is this overall a great setup and strategy? I’m doubtful. (Not just for the org as a whole, but also long-term for Nate himself.)
I think we now have an experimental verification that the document, no matter how impressive, doesn’t always achieve its intended goal.
So, the strategy needs an update.
Three points that might be somewhat revealing:
There was never an ask for reciprocal documents from employees. “Here’s a document describing how to communicate with me. I’d appreciate you sending me pointers on how to communicate with you, since I am aware of my communication issues.” was never considered.
There are multiple independent examples of people in various capacities, including his girlfriend, expressing that their opinions were not valued, and a clear hierarchical model was in play.
The more humble “my list of warnings” was highlighted immediately as justification but never broadcast broadly, and there seems to be no cognizance that it’s not something anyone else would ever take upon themselves to share.
So on one hand… your bullet-points there are maybe pointing in a helpful direction. And I think my overall take right now is “however much effort Nate has previously put into improving on his communications or comms-onboarding, probably he (or MIRI) should put more.”
But, your phrasing here feels a bit like a weird demand for exceptional rigor.
Like, although I think Nate is pretty high on “can feel intense to interact with”, it’s not that weird for a company to have an intense manager, and I’ve never heard of companies-with-intense-managers having this sort of doc at all. And I know a bunch of people who are intense to interact with in regular, interpersonal interactions (i.e. while dating), and they also often don’t have docs explaining that.
So, it feels pretty weird (and not particularly “revealing” of anything) that Nate made a pretty novel type of doc… and didn’t (yet) do some additional followup steps with it.
(I think that “intense” is euphemizing.)
No—the opposite. I was implying that there’s clearly a deeper underpinning to these patterns that any amount of rigor will be insufficient in solving, but my point has been articulated within KurtB’s excellent later comment, and solutions in the earlier comment by jsteinhardt.
I agree; that’s very true. However, this usually occurs in companies that are chasing zero-sum goals. Employees treated in this manner might often resort to a combination of complaining to HR, being bound by NDAs, or biting the bullet while waiting for their paydays. It’s just particularly disheartening to hear of this years-long pattern, especially given the induced discomfort in speaking out and the efforts to downplay, in an organization that publicly aims to save the world.
I wanted to briefly note for now that LW moderators are tracking this thread. Some of us may end up writing more of our thoughts later.
I think it’s good for people to be sharing their experiences. I’m glad Turntrout and KurtB shared their stories. I’m glad So8res responded with his experience/context. I generally feel quite good about people sharing information like this.
I feel a bit wary of any specific recommendations necessarily helping – I think often situations like this are just actually pretty tricky/nuanced and involve a lot more significant tradeoffs than it may seem. (Still seems fine for people to write up their own wishes, or guesses about what would help, I just think it’s just useful to keep in mind that it may be more complicated that it seems at first glance)
I haven’t personally interacted with Nate much, but I’ve had some experiences similar to things Turntrout described with other people I respect. I might have some useful things to say about some of the patterns involved, thought it may take awhile to write up and might end up being a separate top-level post.
For my part, I predict (85%) that this sentiment will age at least somewhat poorly, looking back one year from now. (Not operationalizing it super well, just trying to do better than nothing.)
EDIT: Obviously there’s going to be some nuance, etc. This prediction is a public-facing result of a longer private dialogue I had with other parties.
I think it would have been useful to be informed about Nate’s communication style and reputation here before starting the project, although I doubt this would have changed anyone’s decision to work on the project (I haven’t checked with the others and they might think differently). I think it’s kind of hard to see how bad/annoying/sad this is until you’re in it.
This also isn’t to say that ex post I think joining/doing the project was a bad idea.
Can you say when you were made aware of Nate’s communication style? It sounds here like he didn’t tell you until after you’d started working together, but Jeremy’s comment seems to indicate that you all were informed beforehand?
I think I became most aware in December 2022, during our first set of in-person meetings. Vivek and Thomas Kwa had had more interaction with Nate before this and so might have known before me. I have some memory of things being a bit difficult before the December meetings, but I might have chalked this up to not being in-person, I don’t fully remember.
It was after these meetings that we got the communication guide etc.
Jeremy joined in May 2023, after the earlier members of the team knew about communication stuff and so I think we were able to tell him about various difficulties we’d had.
My negative-feeling meeting with Nate was in July 2022, after which I emailed something like “hey I didn’t consent to that and wouldn’t have agreed to talk if I knew ahead of time” and his reply included something like “sorry, I’ll at least make sure to forewarn future people first, even if they aren’t my formal collaborators; obvious in retrospect.”
Because a) July is well before December and b) the information you just provided, it seems like he does not reliably warn his collaborators.
As Nate clarified at the end of the dialogue, he apparently considered this collaboration to be a “sad/grudging shot” and that context explains “suboptimalities” of your working relationship. But from my limited information and impressions thus far, I don’t think that “neglecting to warn incoming junior researchers” falls under “reasonable suboptimality.” I could imagine him thinking, “hm they’re Vivek’s friends, and Vivek has talked to me, so he probably let them know already.” However, I’d personally consider this to be a serious lapse, given the potential for damage.
(But maybe there’s some more sympathetic perspective here which I’m missing..? I’m happy to update back upwards, given additional clarification or facts.)
Less “hm they’re Vivek’s friends”, more “they are expressly Vivek’s employees”. The working relationship that I attempted to set up was one where I worked directly with Vivek, and gave Vivek budget to hire other people to work with him.
If memory serves, I did go on a long walk with Vivek where I attempted to enumerate the ways that working with me might suck. As for the others, some relevant recollections:
I was originally not planning to have a working relationship with Vivek’s hires. (If memory serves, there were a few early hires that I didn’t have any working relationship with at any point during their tenure.) (If memory serves further, I explicitly registered pessimism, to Vivek, about me working with some of his hires.)
I was already explicitly relying on Vivek to do vetting and make whatever requests for privacy he wanted to, which my brian implicitly lumped in with “give caveats about what parts of the work might suck”.
The initial work patterns felt to me more like Vivek saying “can one of my hires join the call” than “would you like to also do research with my hires directly”, which didn’t trigger my “give caveats personally” event (in part because I was implicitly expecting Vivek to have given caveats).
I had already had technical-ish conversations with Thomas Kwa in March, and he was the first of Vivek’s employees to join calls with me, and so had him binned as already having a sense for my conversation-style; this coincidence further helped my brain fail the “warn Vivek’s employees personally” check.
“Vivek’s hires are on the call” escalated relatively smoothly to “we’re all in a room and I’m giving feedback on everyone’s work” across the course of months, and so there was no sharp boundary for a trigger.
Looking back, I think my error here was mostly in expecting-but-not-requesting-or-verifying that Vivek was giving appropiate caveats to his hires, which is silly in retrospect.
For clarity: I was not at any point like “oops I was supposed to warn all of Vivek’s hires”, though I was at some point (non-spontaneously; it was kinda obvious and others were noticing too; the primary impetus for this wasn’t stemming from me) like “here’s a Nate!culture communication handbook” (among other attempts, like sharing conversation models with mutual-friends who can communicate easily with both me and people-who-were-having-trouble-communicating-with-me, more at their request than at mine).
TurnTrout, if you’re comfortable sharing, I’d be curious to hear more about the nature of your interaction with Nate in July 2022.
Separately, my current read of this thread is something like “I wish it was easier for Nate to communicate with people, but it seems (at least to me) like the community is broadly aware that Nate can be difficult to communicate with & I think his reputation (at least in the AIS circles I’m in) already matches this. Also, it seems like he (at least sometimes) tries to be pretty clear/explicit in warning people about his communication/mentoring style.”
I do think the “giving up” vibe you mention in the original comment is tracking something real. I think the AIS field would benefit if Nate woke up one day and found ways to communicate with people that made them feel more heard/respected/validated and less dismissed/misunderstood/pained. I’m guessing Nate would agree with this, though, and I’m guessing he’s probably tried a lot of things and simply hasn’t found strategies that are effective (and that don’t trade off against other desireada, like honesty or Nate energy/motivation). I’m not sure though—I mostly believe this because other people have told it to me & because it seems like he put in a fair amount of effort/reflection in his communication guide.
I guess my TLDR is something like “I think you’re pointing at a real problem, and it would indeed be great if Nate were better at communicating, but I also feel like there’s something important about Nate’s perspective that feels missing here.”
I disagree on both points. I wasn’t aware before talking with him, and I’d been in alignment and around the Bay for years. Some of my MATS mentees had been totally unaware and were considering engaging with him about something. Peter wasn’t informed before starting a work relationship, and Nate didn’t tell Peter (and maybe Thomas either) before working with them.
I second this—I skimmed part of nate’s comms doc, but it’s unclear to me what turntrout is talking about unless he’s talking about “being blunt”—it sounds that overall there’s something other than bluntness going on, cuz I feel like we already know about bluntness / we’ve thought a lot about upsides and downsides of bluntness people before.
So, I don’t know what actually happened here. But I at least want to convey support for:
“There are ways of communicating other than being blunt that can… unsettlingly affect you [or, at least, some people], which are hard to explain, and their being hard to explain makes it psychologically harder to deal with because when you try to explain it or complain about it people are kinda dismissive.”
(I’m not expressing a strong opinion here about whether Nate should have done something different in this case, or what the best way for Turntrout, Vivek’s team, or others should relate to it. I’m just trying to hold space for “I think there’s a real thing people should be taking seriously as a possibility and not just rounding off to ‘Turntrout should have thicker skin’ or something)
I have some guesses about the details but they’re mostly informed by my interactions with people other than Nate, which give me sort of an existence proof, and I’m wary of speculating myself here without having actually had this sort of conversation with Nate.
I really wish it were possible for this conversation to address what the affected people are coming in with. I suspect (from priors and the comments here) that there are social effects that are at core not located in either Nate or TurnTrout that result in this.
I might reply later, but I want to note that Nate’s comms doc doesn’t really track my (limited) experience of what it feels like to talk with Nate, and so (IMO) doesn’t make great sense as a baseline of “what happened?”.
Ah yeah. I’m a bit of a believer in “introspection preys upon those smart enough to think they can do it well but not smart enough to know they’ll be bad at it”[1], at least to a partial degree. So it wouldn’t shock me if a long document wouldn’t capture what matters.
epistemic status: in that sweet spot myself
+1, was gonna make this comment myself, but TurnTrout said it better.
Was this via public comments or private communications? (Regardless, sucks that this happened to you. But it changes my guesses what sorts of solutions would help)
This was a 1-on-1, in-person chat.
While I am not close to this situation, I felt moved to write something, mostly to support junior researchers and staff such as TurnTrout, Thomas Kwa, and KurtB who are voicing difficult experiences that may be challenging for them to talk about; and partly because I can provide perspective as someone who has managed many researchers and worked in a variety of research and non-research organizations and so can more authoritatively speak to what behaviors are ‘normal’ and what patterns tend to lead to good or bad outcomes. Caveat that I know very little about any internal details of MIRI, but I am still reasonably confident of what I’m saying based on general patterns and experience in the world.
Based on reading Thomas Kwa’s experience, as well as KurtB’s experience, Nate Soares’ behavior is far outside any norms of acceptable behavior that I’d endorse. Accepting or normalizing this behavior within an organization has a corrosive effect on the morale, epsistemics, and spiritual well-being of its members. The morale effects are probably obvious, but regarding epistemics, leadership is significantly less likely to get useful feedback if people are afraid to cross them (psychological safety is an important concept here). Finally, regarding spirit, normalizing this behavior sends a message to people that they aren’t entitled to set boundaries or be respected, which can create far-reaching damage in their other interactions and in their image of themselves. Based on this, I feel very worried for MIRI and think it should probably do a serious re-think of its organizational culture.
Since some commenters brought up academia and the idea that some professors can be negligent or difficult to work with, I will compare Nate’s behavior to professors in CS academia. Looking at what Thomas Kwa described, I can think of some professors who exhibit individual traits in Thomas’ description, but someone who had all of them at once would be an outlier (in a field that is already welcoming to difficult personalities), and I would strongly warn students against working with such a person. KurtB’s experience goes beyond that and seems at least a standard deviation worse; if someone behaved this way, I would try to minimize their influence in any organization I was part of and refuse to collaborate with them, and I would expect even a tenured faculty to have a serious talking-to about their behavior from colleagues (though maybe some places would be too cowardly to have this conversation), and for HR complaints to stack up.
Nate, the best description I can think of for what’s going on is that you have fairly severe issues with emotional regulation. Your comments indicate that you see this as a basic aspect of your emotional make-up (and maybe intimately tied to your ability to do research), but I have seen this pattern several times before and I am pretty confident this is not the case. In previous cases I’ve seen, the person in question expresses or exhibits and unwillingness to change up until the point that they face clear consequences for their actions, at which point (after a period of expressing outrage) they buckle down and make the changes, which usually changes their own life for the better, including being able to think more clearly. A first step would be going to therapy, which I definitely recommend. I am pretty confident that even for your own sake you should make a serious effort to make changes here. (I hope this doesn’t come across as condescending, as I genuinely believe this is good advice.)
Along these lines, for people around Nate who think that they “have” to accept this behavior because Nate’s work is important, even on those grounds alone setting boundaries on the behavior will lead to better outcomes.
Here is an example of how an organization could set boundaries on this behavior: If Nate yells at a staff member, that staff member no longer does ops work for Nate until he apologizes and expresses a credible commitment to communicate more courteously in the future. (This could be done in principle by making it opt-in to do continued ops work for Nate if this happens, and working hard to create a real affordance for not opting in.)
The important principle here is that Nate internalizes the costs of his decisions (by removing his ability to impose costs on others, and bearing the resulting inconvenience). Here the cost to Nate is also generally lower than the cost that would have been imposed on others (inflating your own bike tire is less costly than having your day ruined by being yelled at), though this isn’t crucial. The important thing is Nate would have skin in the game—if he still doesn’t change, then I believe somewhat more that he’s actually incapable of doing so, but I would guess that this would actually lead to changes. And if MIRI for some reason believes that other people should be willing to bear large costs for small benefits to Nate, they should also hire a dedicated staff to do damage control for him. (Maybe some or all of this is already happening… I am not at MIRI so I don’t know, but it doesn’t sound this way based on the experiences that have been shared.)
In summary: based on my own personal experience across many organizations, Nate’s behavior is not okay and MIRI should set boundaries on it. I do not believe Nate’s claim that this is a fundamental aspect of his emotional make-up, as it matches other patterns in the past that have changed when consequences were imposed, and even if it is a fundamental aspect he should face the natural consequences of his actions. These consequences should center on removing his ability to harm others, or, if this is not feasible, creating institutions at MIRI to reliably clean up after him and maintain psychological safety.
Has anyone claimed at this point that Nate yelled at them?
Nate’s an asshole, and this is cult dynamics. Make your wisdom saving throws, folks.
Discussing the problem publicly is the opposite of cult dynamics. (You may be correct about Nate though.)
I’m MIRI’s new research manager and I’d like to report back on the actions we’ve taken inside MIRI in response to the experiences reported above (and others). In fact I joined MIRI earlier this year in part because we believe we can do better on this.
First off, I’d like to thank everyone in this thread for your bravery (especially @KurtB and @TurnTrout). I know this is not easy to speak about and I’d like you to know that you have been heard and that you have contributed to a real improvement.
Second, I’d like to say that I, personally, as well as MIRI the org take these concerns very seriously and we’ve spent the intervening time coming up with internal reforms. Across MIRI research, comms and ops, we want every MIRI staff member to have a safe environment to work in and to not have to engage in any interactions they do not consent to. For my area of responsibility in research, I’d like to make a public commitment to firmly aim for this.
To achieve this we’ve set up the following:
Nate currently does not directly manage any staff. By default, all new research staff will be managed by me (Lisa) and don’t need to interact with Nate. Further, should he ever want to manage researchers at MIRI again, any potential staff wanting to be managed by him shall go through a rigorous consent process and then be given the option of an informed choice on whether they’d like to work with him. This will include sharing of experience reports such as in this thread, conversations with staff who worked with Nate previously as well as access to Nate’s communication handbook. We are also considering adding a new onboarding step which involves a communication norms conversation between Nate and the new staff moderated by a therapist with communications experience. (We are unsure how effective this is, and would trial it)
Second, any new staff working with Nate shall be allowed to first work on a trial period and to be given generous support from MIRI in case of problems (this can include switching their manager, having a designated people manager they can speak to, having a severance agreement in place, as well as speaking with a licensed therapist if desirable).
We will also work on drafting a new internal communications policy which we will expect all our staff including Nate to abide by. We acknowledge that this will likely be vague. Our “path to impact” for this is a hope that this will make it easier for staff to bring up problems, by having a clause in the policy to point to and have less of an insecurity barrier towards concluding a problem is worth bringing up.
We don’t think Nate’s exceptional skill set excuses his behavior, yet we also acknowledge his ability to make unique contributions and want to leverage that while minimising (ideally avoiding) harm. This narrative would feel incomplete without me (Lisa) acknowledging that I do think Nate deeply cares about his colleagues and that the communication is going badly for different reasons.
Finally, I’d like to invite all who have thoughts to share on how to make this change effective or who’d like to privately share about other experience reports to reach out to me here on LessWrong.
I think this discussion has been hard, but I’m glad we had it and I think it will lead to lasting positive change at MIRI.
I’m very happy to read this. I feel lucky that Lisa works at MIRI, and I feel major gratitude to her and everyone else who contributed behind the scenes.
I’m also really glad to see this. Seems like a very positive set of changes. Props to MIRI for taking concrete and decisive action.
I want to say some things about the experience working with Nate, I’m not sure how coherent this will be.
Reflections on working with Nate
I think jsteinhardt is pretty correct when he talks about psychological safety, I think our conversations with Nate often didn’t feel particularly “safe”, possibly because Nate assumes his conversation partners will be as robust as him.
Nate can pretty easily bulldoze/steamroll over you in conversation, in a way that requires a lot of fortitude to stand up to, and eventually one can just kind of give up. This could happen if you ask a question (and maybe the question was confused in some way) and Nate responds with something of a rant that makes you feel dumb for even asking the question. Or often we/I felt like Nate had assumed we were asking a different thing, and would go on a spiel that would kind of assume you didn’t know what was going on. This often felt like rounding your statements off to the dumbest version. I think it often did turn out that the questions we asked were confused, this seems pretty expected given that we were doing deconfusion/conceptual work where part of the aim is to work out which questions are reasonable to ask.
I think it should have been possible for Nate to give feedback in a way that didn’t make you feel sad/bad or like you shouldn’t have asked the question in the first place. The feedback we often got was fairly cutting, and I feel like it should be possible to give basically the exact same feedback without making the other person feel sad/bad/frustrated.
Nate would often go on fairly long rants (not sure there is a more charitable word), and it could be hard to get a word in to say “I didn’t really want a response like this, and I don’t think it’s particularly useful”.
Sometimes it seemed like Nate was in a bad mood (or maybe our specific things we wanted to talk about caused him a lot of distress and despair). I remember feeling pretty rough after days that went badly, and then extremely relieved when they went well.
Overall, I think the norms of Nate-culture are pretty at-odds with standard norms. I think in general if you are going to do something norm-violating, you should warn the people you are interacting with (which did eventually happen).
Positive things
Nate is very smart, and it was clearly taxing/frustrating to work with us much of the time. In this sense he put in a bunch of effort, where the obvious alternative is to just not talk to us. (This is different than putting in effort into making communication go well or making things easy for us).
Nate is clearly trying to solve the problem, and has been working on it for a long time. I can see how it would be frustrating when people aren’t understanding something that you worked out 10 years ago (or were possibly never confused about in the first place). I can imagine that it really sucks being in Nate’s position, feeling the world is burning, almost no one is trying to save it, those who are trying to save it are looking at the wrong thing, and even when you try to point people at the thing to look at they keep turning to look at something else (something easier, less scary, more approachable, but useless).
We actually did learn a bunch of things, and I think most/all of us feel like we can think better about alignment than before we started. There is some MIRI/Nate/Eliezer frame of the alignment problem that basically no one else has. I think it is very hard to work this out just from MIRI’s public writing, particularly the content related to the Sharp Left Turn. But from talking to Nate (a lot), I think I do (partially) understand this frame, I think this is not nonsense, and is important.
If this frame is the correct one, and working with Nate in a somewhat painful environment is the only way to learn it, then this does seem to be worth it. (Note that I am not convinced that the environment needed to be this hard, and it seems very likely to me that we should have been able to have meetings which were both less difficult and more productive).
It also seems important to note that when chatting with Nate about things other than alignment the conversations were good. They didn’t have this “bulldozer” quality, they were frequently fun and kind, and didn’t feel “unsafe”.
I have some empathy for the position that Nate didn’t really sign up to be a mentor, and we suddenly had all these expectations for him. And then the project kind of morphed into a thing where we expected Nate-mentorship, which he did somewhat grudgingly, and assumed that because we kept requesting meetings that we were ok with dealing with the communication difficulties.
I would probably ex post still decide to join the project
I think I learned a lot, and the majority of this is because of Nate’s mentorship. I am genuinely grateful for this.
I do think that the project could have been more efficient if we had better communication, and it does feel (from my non-Nate perspective) that this should have been an option.
I think that being warned/informed earlier about likely communication difficulties would have helped us prepare and mitigate these, rather than getting somewhat blindsided. It would also have just been nice to have some explicit agreement for the new norms, and some acknowledgement that these are not standard communication norms.
I feel pretty conflicted about various things. I think that there should clearly be incentives such that people with power can’t get away with being disrespectful/mean to people under them. Most people should be able to do this. I think that sometimes people should be able to lay out their abnormal communication norms, and give people the option of engaging with them or not (I’m pretty confused about how this interacts with various power dynamics). I wouldn’t want strict rules on communication stopping people like Nate being able to share their skills/knowledge/etc with others; I would like those others to be fully informed about what they are getting into.
I am curious to what extent you or Nate think I understand that frame? And how easy it would be to help me fully get it? I am confused about how confused I am.
I understand why someone might be frustrated in his position, and it’s fine to feel however. However, I want to push back on any implicit advancement of sentiments like “his intense feelings justify the behavior.”[1] The existing discussion has focused a lot on the social consequences of e.g. aggressive and mean behavior. I’ll now take a more pragmatic view.
If you want to convince people of something, you should not severely punish them for talking to you. For example, I’d be more open to hearing Nate’s perspective if he had conducted himself in an even somewhat reasonable manner. As I wrote in my original comment:
Even from a pragmatic “world-saving” perspective, and given Nate’s apparent views, Nate’s behavior still doesn’t make sense to me. It doesn’t seem like he’s making some clever but uncooperative trade whereby he effectively persuades people of true stuff, albeit at (sometimes) large emotional cost to others. It seems more like “relative lack of persuasion, and other people sometimes get hurt (without agreeing to it), and people sometimes become significantly less interested in considering his views.”
I sometimes get frustrated that people still seem to be missing key shard theory insights, even after several conversations. I get frustrated that Nate in particular possibly still doesn’t understand what I was trying to explain in our July 2022 chat. I still do not rant at people or leave them feeling intensely drained. Even if my emotions were more intense, I would still think it pragmatically unwise to have strong negative effects on my employees and colleagues.
Probably you (Peter) did not mean to imply this, in which case my comment will just make the general point.
I’m struck by this:
This might be true, and if true it might be very important. But, outside view, I think the track record of people/organizations claiming things along the lines of “we and we alone have the correct understanding of X, and your only way to understand X is to seek our wisdom” is pretty bad, and that of people/organizations about whom other people say “they and they alone have the correct understanding, etc.” isn’t much better.
I know that MIRI expresses concern about the dangers of spreading their understanding of things that might possibly be used to advance AI capabilities. But if an important thing they have is a uniquely insightful way of framing the alignment problem then that seems like the sort of thing that (1) is very unlikely to be dangerous to reveal, (2) could be very valuable to share with others, and (3) if so shared would (a) encourage others to take MIRI more seriously, if indeed it turns out that they have uniquely insightful ways of thinking about alignment and (b) provide opportunities to correct errors they’re missing, if in fact what they have is (something like) plausible rhetoric that doesn’t stand up to close critical examination.
I think the 2021 MIRI Conversations and 2022 MIRI Alignment Discussion sequences are an attempt at this. I feel like I have a relatively good handle on their frame after reading those sequences, and I think the ideas contained within are pretty insightful.
Like Zvi, I might be confused about how confused I am, but I don’t think it’s because they’re trying to keep their views secret. Maybe there’s some more specific capabilities-adjacent stuff they’re not sharing, but I suspect the thing the grandparent is getting at is more about a communication difficulty that in practice seems to be overcome mostly by working together directly, as opposed to the interpretation that they’re deliberately not communicating their basic views for secrecy reasons.
(I also found Eliezer’s fiction helpful for internalizing his worldview in general, and IMO it is also has some pretty unique insights.)
I haven’t really interacted with Nate in the sorts of contexts getting discussed here. But one of Turntrout’s comments reminded me of some similar experiences I’ve had with other people that I wanted to write up.
I think there are three high level points:
“Relative Forcefulness-of-personality” is variable to be aware of.
“Suddenly having to defend boundaries you didn’t expect to have to” is a particular way that it can be difficult/scary interacting with someone more forceful than you. (Corollary: it’s scarier to interact with forceful people who do a lot of unpredictable things)
“Forcefully telling people how they think” can feel particularly violating, to somewho is less self-assured, because it can leave them doubting their sanity, and undermining their ability to figure out what they want.
I feel sold on “these are things to be aware of.” I don’t necessarily prescribe any particular takeaway with them.
I think rather than write a giant effort-comment I’m going to write pieces at a time. Fleshing out point #1 a bit:
People vary in how forceful a personality they are. I don’t think there’s a single “correct” amount of forcefulness to be, but whenever there’s a mismatch it can leave some people feeling steamrolled. This varies all the way down to “both people are very timid / guess-culture-y, but one is slightly more ask-culture-y” to includes “one of the people is willing to escalate to physical violence and the other is only willing to yell and scream.”
I think there’s also a lot of mini-variables here. There’s “how confident is someone in what they want?” or “in what they believe?”. There’s “how much has a person been socialized to try to prioritize the needs of others, vs their own needs?”. Or “do they feel entitled to get angry? Do they feel entitled to walk away from a conflict vs sit there and deal with it?”. How much are they willing to escalate to get what they want?
So the first thing here is “it’s at least possible for people to have very different forcefulnesses, and different “self-securenesses”. And it may not be obvious to Alex and Bob how much Alex is persistently steamrolling Bob.
Is that Alex’s problem? I dunno. I have previously argued with Alexes that it should be their problem and they were like “I think Bob should grow more of a spine, I should be allowed to say things I think are true without having to worry about Alex’s feelings.”
I think, regardless of whether you locate “responsibility” in any particular place, I think it’s useful at the very least for Alex to be modeling that they might be having a stronger effect than he naively realizes (and I’d guess worth spending at least some time thinking about alternate strategies or ways of phrasing things), and it’s useful for Bob to think of “grow a spine” or “leave” or “change his relational stance to Alex” as some of the options available to him.
...
I’ll leave it there for now.
Okay, now shifting to give more details about #3:
“Forcefully telling people how they think”
Here’s an example situation. One day, Alex, Bob and me were all having a negotiation, trying to figure out what to do about Dave. We all agreed Dave had done something wrong. Alex and Bob both thought Dave was… something like “overwhelmingly in the wrong, and had defected first.” I thought that we’d all been kinda complicit in an escalation spiral, and I cut Dave more slack.
At a point when Alex, Bob and I were all in a room talking about Dave, and I was pointing out a way that Bob had escalated things with Dave.
Alex said, forcefully, with a bit of edge in his voice, “Ray is only saying that because he has a pathological need to listen to Dave and [I can’t remember the exact phrasing here].
And...
...I think Alex was at least moderately right. There is some way in which I feel compelled to take people’s stories at face value, and be kind of a bleeding heart for them, and put in extra work trying to cater to them. Often at my own expense and the expense of other important stuff I care about. I think this has in fact been the wrong call, and I’ve made significant updates over the years about being more stone-hearted about it.
But, also, in this case, I’m pretty sure this was at most 25% of what was going on (at least in this conversation). I remember some specific ways I updated about Dave that week, and I trust my deliberate introspection about it (even if I don’t trust all of my background-process-introspection).
This was all during a pretty tense time period, when we were all kinda exhausted.
A few things that stick out here:
Alex spoke really confidently. I think in general this triggers some kind of “I should listen to what this person is saying” behavior. I also do respect their opinions a lot (often more than my own), so part of me was thinking “I dunno am I the crazy one here? Am I wrong about why I’m thinking this? I’m pretty sure I’m not wrong.” I felt like my own sense-of-self was under siege.
I expected Bob also to be listening to what Alex said, which meant my ability to speak and be-heard within this group also felt under siege.
A thing about Alex was that he was often willing to escalate in ways that I wouldn’t have. Like, we were having this conversation because he wanted to escalate conflict with Dave in ways that seemed excessive to me. This meant I wasn’t sure what would happen if I ended up in direct conflict with Alex. I think Alex had a model of exactly when-and-how he’d escalate that seemed clear and fair to him, but I didn’t understand it. So I felt “an unknown amount of threat” that was anxiety inducing.
I think Alex was, in part, protecting Bob from me. I also speak more forcefully/confidently than average. I think it seemed to Alex like I was making it harder for Bob to think through and stand by their own opinions.
Relatedly: later on when I accused Alex of speaking with edge-in-his voice that conveyed a willingness to escalate in scary ways, he noted that I also speak with edge in my voice a fair amount, probably more than Alex. This does seem true on reflection.
I am fairly uncertain how to think about all of this. It’s noteworthy that I think Alex was wrong about my *overall* motivations here, but wasn’t wrong about at least part of my motivations. This makes for an annoying situation where he has a reasonably justified story of why my account-of-myself is untrustworthy (which makes it hard to correct), but, is still wrong in some ways.
The thing that feels most confusing here to me is how to spell out “why exactly this feels so bad.” I felt like my epistemics and sense-of-self-direction were under assault. Was that “real”, or was it just a feeling, and the problem is located inside my feelings rather than inside Alex’s speech-actions?
Digging into this feels like “the hard part”. I’ll leave this here for now again.
I guess another thing to note is that “people telling you that you are wrong about how you think” can also be an important part of breaking out of wrong, sticky narratives you have about yourself.
Perhaps annoyingly: many of the people who I have found very helpful for learning to “see the water I was swimming in, and take it as object” seem also actively destabilizing for some people around them. (i.e. I’ve gotten a lot of value from Brent, Ziz, Vassar and Geoff Anders. In each of those cases I didn’t actually get too close for long, but I know other people who did and had various flavors of bad experience).
Habryka’s gloss on all this is “telling people they are wrong about what they think is high variance and should be treated as risky, but also has important upside.” I feel like “high variance” is too positive a spin on it, but there’s something important there.
I’d like to point out that of these three things, #3 is one (and, note, the only one) which is traditionally frowned upon, according to commonly accepted rules of conversational conduct, independently of any views about how “forceful” one ought to be, etc. This seems to me to be instructive. (We have many words/phrases for it, too: “Bulverism”, “psychoanalyzing your conversation partner”, etc. These are not laudatory terms!)
Thanks for sharing these factors.
To add: different orientations to the conversation can produce different forcefulness-of-personality. In the first half of my chat with Nate, I was rather cooperative and truth-seeking, and it mostly felt like Nate was steamrolling me. In the second half, I oriented towards it as an adversarial sparring match with someone who was trying to beat me, and began calling out a bunch of (IMO) shaky claims, and the conversation felt pretty even.
Note I have never met Nate, but have taken some amount of value from his writings. I have met KurtB briefly around a year ago and enjoyed some excellent conversations at the time. I can’t remember if I’ve met TurnTrout but we’ve certainly interacted with medium success online. This comment is not meant to add any evidential weight to any part of anyone’s previous statements.
I am reminded by some of these comments and threads (especially the one with TurnTrout and KurtB) of the ‘super chicken’ model—researchers selecting chickens for high egg output inadvertently produced high levels of conspecific violence and aggression (it turned out to be easier to be best-in-cohort by putting down other locals than by actually laying more eggs). I think maybe Eliezer wrote about this somewhere.
One hypothesis might be that Nate is simultaneously good at laying eggs and also prone (not necessarily deliberately) to pecking conspecifics.