I’ve been busy this week and not actually soaked in all the details from the newer Nonlinear response post, or this one (I have skimmed both). I work at Lightcone. I wasn’t heavily involved in this investigation, but I helped a bit from the sidelines and generally supported it and probably deserve some small portion of whatever credit/blame end up going to Ben/Habryka when the dust settles.
I disagree with much of the framing here. I think I am also more skeptical than you about how much Nonlinear’s counterclaims exonerate them (Many of their own statements skeeve me out in a way that isn’t very dependent on Alice’s veracity)
But, one update I did make reading this post:
“I think it was probably still correct to ship quickly, given what Ben knew at the time, but, I think it’d have been better to avoid editorializing. i.e. stick to propagating facts.”
The reason I still am in favor of not delaying a week is that I think the risks of the adversarial slowdown, or threatening/pressuring witnesses into silence, are real, and this post doesn’t engage with them enough IMO.
I agree with the failure modes you list here of shipping quickly. If we were going to trade off those failure modes for dealing with adversarial slowdown / pressuring, I think the original post should have been framed around “we are here to share information, not to deliver a verdict”, and it should have been designed to emphasize that, for two reasons:
it’s a better policy for a community having a healthy OODA loop. Having a post that just lays out observations before trying to orient or decide seems epistemically good to me.
I think “we were serving as the prosecution, not trying to be the jury or judge” is a reasonable, coherent frame. So there’s a natural role for “provide evidence without dictating judgment”. And, in this sort of situation where “vague social judgment” is the primary outcome, seems maybe better to steer away from it.
I don’t think these reasons are overwhelming, because I also think there was something important for Ben to share about his overall epistemic state, both for making things concrete, and synethisizing some small bits of info from various conversations that are hard to convey. But it’d have been better to stop around the “sharing my epistemic state” section of the post, before the “general comments from me” section.
I think this was an achievable thing for me to notice and advocate more for, at the time. (I think I maybe had advocated for something similar, briefly, modeled after the Intentional Insights takedown post which avoided having an editorial throughline, and then coauthors gave more opinionated takes in the comments. But, I didn’t push for it very hard).
I think I am also more skeptical than you about how much Nonlinear’s counterclaims exonerate them
I mention this in the post and realize this is a frustrating frame for the Lightcone people who worked hard on this story, but I really just don’t care a ton about Nonlinear qua Nonlinear. It’s a small charity org with an unconventional structure working in a general area (AI safety) a lot of rationalists see as important and I have ambiguous feelings on in terms of the efficacy of charity work. I don’t have a lot of weight as to where people should land on whether their claims “exonerate” them in a true sense, particularly because the stakes feel a lot more like “roommate drama” stakes than “FTX” ones to me.
What I do care very much about is that the rationalist/EA community not fall into the same callout/dogpiling/”cancel” cultural traps I’ve seen repeat in so many other subcultures. Spending six months to gather only negative information about someone—particularly someone in your own community, where your opinion will carry a lot of weight—before presenting it in public is bad, full stop. It works in a courtroom because there is a judge there, but the court of public opinion demands different norms.
Doing so and then being in such a hurry to present it that you won’t even pause when someone not under investigation brings in hard evidence against one of your claims and tells you you’re making a mistake? That only compounds the issue.
the risks of the adversarial slowdown, or threatening/pressuring witnesses into silence
Part of this feels like a byproduct of having spent six months gathering negative information about them. You all were in an extraordinarily adversarial frame towards them, where my sense is that you all (at least Ben) emotionally felt they were something akin to monsters wearing human skinsuits, some sort of caricature of cartoon villains. From my own outside-community view, you all seem like decent, flawed people with the same broadly praiseworthy, somewhat flawed philosophy.
More than that, though: the witnesses were already not silent. They’d spent a year being anything but silent. They’d spent the better part of six months feeding information to Ben, who had it and could do what he felt he ought with it independent of Nonlinear’s actions. “Please give us a week to present evidence” paired with reassurance that they’re not trying to stop you from publishing altogether contains a clear hard deadline with a clear request and gives no reason to indicate an indefinite delay.
I push towards publication a lot of things a lot of people would really rather we not publish (on BARPod). I don’t have quite the proximity to them you guys had to Nonlinear (which goes both ways—you were more reachable by whatever their response was, but you also shared a great deal of context and had a lot more room for cooperation). They can’t do anything to indefinitely delay publication. When we’re satisfied with the story, we put it up. It is not in their control. When it comes to the publication of your own piece on your own site, you hold all the cards.
So I have more objections to some of this, but I maybe want to take a moment to say:
I do really appreciate that you are here trying to push for good epistemic standards. I definitely think each of the considerations you’re raising are really important. I don’t feel that confident that we made the right call, and I think each of the points you’re making should be a pretty strong default that you need a really good reason to deviate from.
I think you’re wrong about specific points like “Nonlinear couldn’t do anything to indefinitely delay publication” and “Ben had the info and could do what he felt he ought independent of Nonlinear’s actions.”
But, mostly right now it feels like you have one really strong/clear frame of how to do truthseeking. I think “investigative journalism ethics” is one particular frame, but neither the only frame for collective truthseeking nor for “figuring out how a community can/should protect itself from manipulative people.”
I can totally buy deciding, in a few weeks, that, yep, Ben fucked up here. And fwiw I also don’t have any objection to you writing your post now rather than later (I thought that was a pretty weird objection on Habryka’s part, given the circumstances). But, I do wish you were putting more effort into asking “is my conception of truthseeking and set of tradeoffs actually right in all circumstances?”.
The fact that it’s not material to you what’s up with Nonlinear and whether they were bad, feels like it’s missing a major part of the conversation.
Right now tensions are already high and I’m not sure how achievable it is to have a real conversation about it in the immediate future. I’m also just pretty busy right now with unrelated stuff. But fwiw I’d be interested in doing a dialogue with you about that. (I do think that format is somehow better than comment sections at maintaining mutual truthseeking vibe)
I appreciate this response and would love to dive into it more. I’m only loosely familiar with the dialogue format on this site but am definitely game, though I’d request an asynchronous one since I prefer having time to gather my thoughts and maintaining a bit of flexibility around each of our schedules.
I’m not sure what you mean here? If LC had said “ok, we’ll consider all the evidence you provide by [datetime] and update our post as we see appropriate” what are you worried about?
threatening/pressuring witnesses into silence
This seems possible to address with precommitments? I think LC and the sources could have agreed on and included something like “to protect our sources from retaliation aimed at keeping the serious allegations in this post from becoming public, we have agreed with them that from the time we first share a draft with NL we will not honor any requests our sources make to withdraw claims.” And then made sure NL knows this and that LC is serious about it.
As far as I understand Ben seemed to have made a precommitment to Alice and Chloe to publish at a certain date. The idea of them not publishing at that point seems to already break a precommitment and make future precommitments weaker.
I find the idea of precommiting not to update on Alice saying “When it comes to event XY I forgot to mention facts Y and Q” not easy to uphold.
Ben seemed to have made a precommitment to Alice and Chloe to publish at a certain date
Sorry, I’m trying to talk about how to do this sort of thing in the future and what options were available to someone in Ben’s position. If Ben committed to Alice and Chloe that he’d publish on a certain date (and it’s not clear to me he did, or if so how far out he did) that was a mistake, and then one seriously compounded by not starting ‘adversarial’ draft review far enough in advance of that date.
What I’d like to see from future people in Ben’s position is something like:
A: I can’t publish this list of accusations without letting the other side raise potential counterevidence
B: I’m concerned about retaliation during the period when the accused knows this is coming but before it’s public, with the goal of getting me to retract
A: I’m willing to commit to not honoring any requests you make to withdraw claims between now and publication, and to make the accused aware of this, so they won’t benefit from retailiation
I think this handles the retaliation-against-sources-to-prevent-publication concern?
I find the idea of precommiting not to update on Alice saying “When it comes to event XY I forgot to mention facts Y and Q” not easy to uphold.
Maybe think of it like launching an ICBM you can’t remotely disable. The decision for the sources analogous to launching would be when they tell Ben that they’re ok with their claims in the draft as is, and they’re ready to send it to NL. After that point Ben can decide to drop claims or add things he learns from NL, but the sources do not get to provide additional input until after publication.
The decision of whether or not to launch a ICBM is a binary one.
Take the issue of the illegal drugs. Imagine that Alice first version of the story was as unspecific as it’s written. One day in Kate writes in and says “It’s ADHD medication + antibiotics and not really illegal material”. Then Alice sends an email saying “”It’s ADHD medication + antibiotics and not really illegal material”.
If Ben then publishes a story saying “It’s Alice position that it’s illegal drugs and Kat’s position that Alice wasn’t asked to bring over illegal drugs” Ben would likely get pushback for that. In an actual court case, it can be argued that Ben misrepresents the evidence available to him.
It sounds like you’re describing a situation like:
A: [to B, in advance] they asked me to bring drugs into the country illegally.
C: [to B, in fact checking] yes, but the drugs were over-the-counter medications and we thought it was legal.
A: [to B, after fact checking has begun] C told me about the drug thing and asked me to retract my claim. I didn’t lie to you, but I probably should have mentioned that they were OTC and C didn’t know they were legal.
With this strategy B would ignore that A agrees with C’s clarification. And would also ignore it if instead it had been:
A: [to B, after fact checking has begun] C told me about the drug thing and asked me to retract my claim, but is thinking of a different time; I was taking about the crack.
If it’s important to A that C cannot retaliate against them to shut them up and they’ve decided to use the precommittment process I’m proposing, this requires giving up some flexibility. In this case the time for B to get clarity from A on exactly what they’re claiming would have to be before starting the adversarial fact checking process. Once it has started, all B can do is consider what they heard from A before fact checking and what they’re hearing from C now and choose between keeping the claim, weakening it, clarifying it, or dropping it.
Note that weakening it is quite tricky: you can’t do it in a way that makes it sound like A is claiming something different than they already endorsed before fact checking, so if you do want to keep the claim I think clarifying it will often make more sense. In this scenario I think I’d go with something like “A told me C asked them to bring drugs into the country illegally. During fact checking C agreed that it wouldn’t have been legal, but also said they were over-the-counter medications and at the time they made the request they believed bringing them in was legal.” And then potentially adding “Due to the precommitments to our sources I described above, I have not asked A whether they agree with C’s description of the situation, but even if C is correct I think it demonstrates a reckless attitude toward the sensitive matter of bringing controlled substances between jurisdictions.”
I’ve been busy this week and not actually soaked in all the details from the newer Nonlinear response post, or this one (I have skimmed both). I work at Lightcone. I wasn’t heavily involved in this investigation, but I helped a bit from the sidelines and generally supported it and probably deserve some small portion of whatever credit/blame end up going to Ben/Habryka when the dust settles.
I disagree with much of the framing here. I think I am also more skeptical than you about how much Nonlinear’s counterclaims exonerate them (Many of their own statements skeeve me out in a way that isn’t very dependent on Alice’s veracity)
But, one update I did make reading this post:
“I think it was probably still correct to ship quickly, given what Ben knew at the time, but, I think it’d have been better to avoid editorializing. i.e. stick to propagating facts.”
The reason I still am in favor of not delaying a week is that I think the risks of the adversarial slowdown, or threatening/pressuring witnesses into silence, are real, and this post doesn’t engage with them enough IMO.
I agree with the failure modes you list here of shipping quickly. If we were going to trade off those failure modes for dealing with adversarial slowdown / pressuring, I think the original post should have been framed around “we are here to share information, not to deliver a verdict”, and it should have been designed to emphasize that, for two reasons:
it’s a better policy for a community having a healthy OODA loop. Having a post that just lays out observations before trying to orient or decide seems epistemically good to me.
I think “we were serving as the prosecution, not trying to be the jury or judge” is a reasonable, coherent frame. So there’s a natural role for “provide evidence without dictating judgment”. And, in this sort of situation where “vague social judgment” is the primary outcome, seems maybe better to steer away from it.
I don’t think these reasons are overwhelming, because I also think there was something important for Ben to share about his overall epistemic state, both for making things concrete, and synethisizing some small bits of info from various conversations that are hard to convey. But it’d have been better to stop around the “sharing my epistemic state” section of the post, before the “general comments from me” section.
I think this was an achievable thing for me to notice and advocate more for, at the time. (I think I maybe had advocated for something similar, briefly, modeled after the Intentional Insights takedown post which avoided having an editorial throughline, and then coauthors gave more opinionated takes in the comments. But, I didn’t push for it very hard).
I mention this in the post and realize this is a frustrating frame for the Lightcone people who worked hard on this story, but I really just don’t care a ton about Nonlinear qua Nonlinear. It’s a small charity org with an unconventional structure working in a general area (AI safety) a lot of rationalists see as important and I have ambiguous feelings on in terms of the efficacy of charity work. I don’t have a lot of weight as to where people should land on whether their claims “exonerate” them in a true sense, particularly because the stakes feel a lot more like “roommate drama” stakes than “FTX” ones to me.
What I do care very much about is that the rationalist/EA community not fall into the same callout/dogpiling/”cancel” cultural traps I’ve seen repeat in so many other subcultures. Spending six months to gather only negative information about someone—particularly someone in your own community, where your opinion will carry a lot of weight—before presenting it in public is bad, full stop. It works in a courtroom because there is a judge there, but the court of public opinion demands different norms.
Doing so and then being in such a hurry to present it that you won’t even pause when someone not under investigation brings in hard evidence against one of your claims and tells you you’re making a mistake? That only compounds the issue.
Part of this feels like a byproduct of having spent six months gathering negative information about them. You all were in an extraordinarily adversarial frame towards them, where my sense is that you all (at least Ben) emotionally felt they were something akin to monsters wearing human skinsuits, some sort of caricature of cartoon villains. From my own outside-community view, you all seem like decent, flawed people with the same broadly praiseworthy, somewhat flawed philosophy.
More than that, though: the witnesses were already not silent. They’d spent a year being anything but silent. They’d spent the better part of six months feeding information to Ben, who had it and could do what he felt he ought with it independent of Nonlinear’s actions. “Please give us a week to present evidence” paired with reassurance that they’re not trying to stop you from publishing altogether contains a clear hard deadline with a clear request and gives no reason to indicate an indefinite delay.
I push towards publication a lot of things a lot of people would really rather we not publish (on BARPod). I don’t have quite the proximity to them you guys had to Nonlinear (which goes both ways—you were more reachable by whatever their response was, but you also shared a great deal of context and had a lot more room for cooperation). They can’t do anything to indefinitely delay publication. When we’re satisfied with the story, we put it up. It is not in their control. When it comes to the publication of your own piece on your own site, you hold all the cards.
So I have more objections to some of this, but I maybe want to take a moment to say:
I do really appreciate that you are here trying to push for good epistemic standards. I definitely think each of the considerations you’re raising are really important. I don’t feel that confident that we made the right call, and I think each of the points you’re making should be a pretty strong default that you need a really good reason to deviate from.
I think you’re wrong about specific points like “Nonlinear couldn’t do anything to indefinitely delay publication” and “Ben had the info and could do what he felt he ought independent of Nonlinear’s actions.”
But, mostly right now it feels like you have one really strong/clear frame of how to do truthseeking. I think “investigative journalism ethics” is one particular frame, but neither the only frame for collective truthseeking nor for “figuring out how a community can/should protect itself from manipulative people.”
I can totally buy deciding, in a few weeks, that, yep, Ben fucked up here. And fwiw I also don’t have any objection to you writing your post now rather than later (I thought that was a pretty weird objection on Habryka’s part, given the circumstances). But, I do wish you were putting more effort into asking “is my conception of truthseeking and set of tradeoffs actually right in all circumstances?”.
The fact that it’s not material to you what’s up with Nonlinear and whether they were bad, feels like it’s missing a major part of the conversation.
Right now tensions are already high and I’m not sure how achievable it is to have a real conversation about it in the immediate future. I’m also just pretty busy right now with unrelated stuff. But fwiw I’d be interested in doing a dialogue with you about that. (I do think that format is somehow better than comment sections at maintaining mutual truthseeking vibe)
I appreciate this response and would love to dive into it more. I’m only loosely familiar with the dialogue format on this site but am definitely game, though I’d request an asynchronous one since I prefer having time to gather my thoughts and maintaining a bit of flexibility around each of our schedules.
Yep reasonable. I’m busy this week but if it still feels promising next week can followup then.
I’m not sure what you mean here? If LC had said “ok, we’ll consider all the evidence you provide by [datetime] and update our post as we see appropriate” what are you worried about?
This seems possible to address with precommitments? I think LC and the sources could have agreed on and included something like “to protect our sources from retaliation aimed at keeping the serious allegations in this post from becoming public, we have agreed with them that from the time we first share a draft with NL we will not honor any requests our sources make to withdraw claims.” And then made sure NL knows this and that LC is serious about it.
As far as I understand Ben seemed to have made a precommitment to Alice and Chloe to publish at a certain date. The idea of them not publishing at that point seems to already break a precommitment and make future precommitments weaker.
I find the idea of precommiting not to update on Alice saying “When it comes to event XY I forgot to mention facts Y and Q” not easy to uphold.
Sorry, I’m trying to talk about how to do this sort of thing in the future and what options were available to someone in Ben’s position. If Ben committed to Alice and Chloe that he’d publish on a certain date (and it’s not clear to me he did, or if so how far out he did) that was a mistake, and then one seriously compounded by not starting ‘adversarial’ draft review far enough in advance of that date.
What I’d like to see from future people in Ben’s position is something like:
I think this handles the retaliation-against-sources-to-prevent-publication concern?
Maybe think of it like launching an ICBM you can’t remotely disable. The decision for the sources analogous to launching would be when they tell Ben that they’re ok with their claims in the draft as is, and they’re ready to send it to NL. After that point Ben can decide to drop claims or add things he learns from NL, but the sources do not get to provide additional input until after publication.
The decision of whether or not to launch a ICBM is a binary one.
Take the issue of the illegal drugs. Imagine that Alice first version of the story was as unspecific as it’s written. One day in Kate writes in and says “It’s ADHD medication + antibiotics and not really illegal material”. Then Alice sends an email saying “”It’s ADHD medication + antibiotics and not really illegal material”.
If Ben then publishes a story saying “It’s Alice position that it’s illegal drugs and Kat’s position that Alice wasn’t asked to bring over illegal drugs” Ben would likely get pushback for that. In an actual court case, it can be argued that Ben misrepresents the evidence available to him.
It sounds like you’re describing a situation like:
With this strategy B would ignore that A agrees with C’s clarification. And would also ignore it if instead it had been:
If it’s important to A that C cannot retaliate against them to shut them up and they’ve decided to use the precommittment process I’m proposing, this requires giving up some flexibility. In this case the time for B to get clarity from A on exactly what they’re claiming would have to be before starting the adversarial fact checking process. Once it has started, all B can do is consider what they heard from A before fact checking and what they’re hearing from C now and choose between keeping the claim, weakening it, clarifying it, or dropping it.
Note that weakening it is quite tricky: you can’t do it in a way that makes it sound like A is claiming something different than they already endorsed before fact checking, so if you do want to keep the claim I think clarifying it will often make more sense. In this scenario I think I’d go with something like “A told me C asked them to bring drugs into the country illegally. During fact checking C agreed that it wouldn’t have been legal, but also said they were over-the-counter medications and at the time they made the request they believed bringing them in was legal.” And then potentially adding “Due to the precommitments to our sources I described above, I have not asked A whether they agree with C’s description of the situation, but even if C is correct I think it demonstrates a reckless attitude toward the sensitive matter of bringing controlled substances between jurisdictions.”