Software engineer at the Nucleic Acid Observatory in Boston. Speaking for myself unless I say otherwise.
jefftk
Karma: 18,964
The motivating example for this post is whether you should say “So, I actually checked with some of their former employees, and if what they say and my corresponding calculations are right, they actually only saved 138 puppies”, with Quinn arguing that you shouldn’t say it because saying it has bad consequences. The problem is, saying this has very clearly good consequences, which means trying to use it as a tool for figuring out what you think of appeals to consequences sets up your intuitions to confuse you.
(It has clearly good consequences because “how much money goes to PADP right now” is far less import than “building a culture of caring about the actual effectiveness of organizations and truly trying to find/make the best ones”. Plus if, say, Animal Charity Evaluators trusted this higher number of puppies saved and it had lead them to recommend PADP as I’ve if their top charities, that that would mean displacing funds that could have gone to more effective animal charities. The whole Effective Altruism project is about trying to figure out how to get the biggest positive impact, and you can’t do this if you declare discussing negative information about organizations off limits.)
The post would be a lot clearer if it had a motivating example that really did have bad consequences, all things considered. As a person who’s strongly pro transparency is hard for me to come up with cases, but there are still contexts where I think it’s probably the case. What if Carter were a researcher who had run a small study on a new infant vaccine and seen elevated autism rates on the experimental group. There’s an existing “vaccines cause autism” meme that is both very probably wrong and very probably harmful, which means Carter should be careful about messaging for their results. Good potential outcomes include:
-
Carter’s experiment is replicated, confirmed, and the vaccine is not rolled out.
-
Carter’s experiment fails to replicate, researchers look into it more, and discover that there was a problem in the initial experiment / in the replication / they need more data / etc
Bad potential outcomes include:
Headlines that say “scientists finally admit vaccines do cause autism”
Because of the potential harmful consequences of handling this poorly, Carter should be careful about how they talk about their results and to who. Trying to get funding to scale up the experiment, making sure the FDA is aware, letting other researchers know, etc, all are beneficial and have good consequences. Going to the mainstream media with a controversial sell-lots-of-papers story, by contrast, would have predictably bad consequences.
When talking with friends or within your field it’s hard to think of cases where you shouldn’t just say the interesting thing you’ve found, while with larger audiences and in less truth-oriented cultures you need to start being more careful.
EDIT: expanded this into https://www.jefftk.com/p/appeals-to-consequences
-
I’m trying to support two complementary points:
-
The norm I’ve been pushing of sharing things with EA organizations ahead of time is only intended for cases where you have a neutral or better relationship with the organization, and not situations like this one where there are allegations of mistreatment, or you don’t trust them to behave cooperatively.
-
A threat to sue if changes are not made to the text of the post is not cooperative.
-
Crossposted from the EA Forum
Thanks for writing this! I’d been putting something together, but this is much more thorough. Here are the parts of my draft that I think still add something:
I’m interested in two overlapping questions:
Should Ben have delayed to evaluate NL’s evidence?
Was Nonlinear wrong to threaten to sue?
While I’ve previously advocated giving friendly organizations a chance to review criticism and prepare a response in advance, primarily as a question of politeness, that’s not the issue here. As I commented on the original post, the norm I’ve been pushing is only intended for cases where you have a neutral or better relationship with the organization, and not situations like this one where there are allegations of mistreatment or you don’t trust them to behave cooperatively. The question here instead is, how do you ensure the accusations you’re signal-boosting are true?
Here’s my understanding of the timeline of ‘adversarial’ fact checking before publication: timeline. Three key bits:
LC first shared the overview of claims 3d before posting.
LC first shared the draft 21hr before posting, which included additional accusations
NL responded to both by asking for a week to gather evidence that they claimed would change LC’s mind, and only threatened to sue when LC insisted on going ahead without hearing their side of the story.
Overall, it does look to me like Ben should have waited. That he was still learning his post had additional false accusations right up to publication, including one so close to publication that he didn’t have time to address it, meant he should have expected that if he didn’t delay it would go to publication containing additional false claims. And Ben seems to have understood this, and told NL two days before that “I do expect you’ll be able to show a bunch of the things you said” and “I did update from things you shared that Alice’s reports are less reliable than I had thought”. Additionally, three days (some of which NL was traveling for) is not long to rebut such a long list of accusations, and some of the accusations were shared with NL less than a day in advance of publishing.
Given that I think it’s clear Ben should have given NL some time to assemble conflicting evidence, do I also need to conclude that it was ok for NL to threaten to sue if he didn’t? That is something I really don’t want to do: I’m not a fan of how strict defamation law is, and I think it’s really valuable that it’s almost never used in our community. If we instead had a culture where everyone is being super careful never to say anything that they would lose a suit over I think we’d see a tremendous chilling effect where many true and important things would never come out. This is especially important around allegations of abusive behavior, where proving the truth of anything can be quite difficult and abusers have often taken advantage of this.
On the other hand, defamation law at its best deters people from saying false things, including fact checking before republishing others’ claims, especially in cases where false statements are likely to have strong personal and professional impacts. NL threatening a suit here seems to me like the law used correctly even given the tighter scope I’d like to see for defamation law, and while it’s still not something I would do and I’d be sad if it became common in our community, I don’t think NL was actually wrong to do it.
Note that I’m not making an overall “Nonlinear: yes or no” judgement here; please don’t interpret this post as a view on whether NL mistreated their employees or on how trustworthy their former employees are. I’ve only been looking at the two questions above: should Ben have waited for additional evidence, and was it an unreasonable escalation for NL to threaten to sue.
A few thoughts about how future posts like LC’s could go better:
Be clear about voice. Ben’s post often clarifies that he’s relaying information (“Alice reported that”) but at other times reports Alice or Chloe’s perspective as his own (“nobody in the house was willing to go out and get her vegan food, so she barely ate for 2 days”). The latter would be warranted if Ben had validated the claims he’s relaying, but my understanding is this was instead just a bit sloppy.
If you have your facts straight you don’t have to give the people you’re writing about a chance to look over a draft and point out errors, and in cases where, for example, people are telling the story of abuse that happened to them that’s often a reasonable decision. On the other hand, especially if you’re reporting on a situation you weren’t a part of, I think you’re really very likely to have some mistakes. Sharing your post’s claims section (you don’t have to share parts of the post that don’t make new claims) for review is really hard to beat as a way of avoiding publishing false claims.
How much time to give for assembling evidence depends on how many claims there are, and I think you can do some prioritization. You can ask if they can provide evidence for some of the places they think you’re most clearly wrong, and if they can’t do a good job with this given a few days I think you’re justified in assuming their claims of counter-evidence are bluster.
When giving time to fact check and gather evidence, get agreement on deadlines up front so things don’t get pushed back repeatedly: “I’ll review anything you get me before EOD Thursday; does that work?” While you should sort out all your claims before starting this process, if you want to add new claims after then you need to push the deadline out to give them more time to respond.
I really like Michael Plant’s comment on a previous post about issues with an EA org (which @Habryka also worked on), responding to someone who thought the post was “charitable and systematic in excess of reasonable caution” and objected to the “fully comprehensive nature of the post and the painstaking lengths it goes to separate out definitely valid issues from potentially invalid ones”:
I take your point as “aren’t we being too nice to this guy?” but I actually really like the approach taken here, which seems extremely fair-minded and diligent. My suspicion is this sort of stuff is long-term really valuable because it establishes good norms for something that will likely recur in future. I’d be much more inclined to act with honesty if I believed people would do an extremely thorough public investigation into everything I’d said, rather than just calling me names and walking away.
If you think that NL was overall in the right then of course LC should have been more careful, but even if you think NL is run by horrible people I think you still should come to that conclusion. This combination of posts and overwhelming amounts of conflicting screenshots and stories seems to have left most of the community not knowing what to think. A post from LC that led with airtight accusations and then included supplementary material for fuzzier issues would have been much more effective towards its goals, enough so that I think more hours spent in ‘adversarial fact checking’, resolving conflicting claims, would have been far more effective per hour than the average for this project.
I almost always post immediately after writing things, since that’s how my motivation works, but in this case I sent my post out for review. I got some good feedback from several people, but now that the top-level post has opened the conversation about appropriate process in situations like these I think talking in public makes more sense.
There are enough EA orgs that I know something about (and that other people know I know something about) that I think the number of bits I was leaking here was pretty low?
Another thing that’s not visible is that I sent Lincoln an email linking to this thread, which I expect is why he jumped in with more context. I really appreciate him doing so, and don’t want him and Wave to end up worse off than in a world in which he’d stayed quiet.
I wrote a response to this here: https://www.jefftk.com/p/theres-lots-more-to-do
Who do you currently live with most of the time? (Alone, with parents or other guardians, with partner and/or children, with roommates.)
My house is currently: me, wife, daughter, sister, another sister, mother, father. I put “with partner and/or children”, but that doesn’t seem like a good fit.
Please give the score you got on your most recent PROFESSIONAL, SCIENTIFIC IQ test
This makes me feel like I should have an IQ number to put here? Is that a thing people usually have?
Place your right hand firmly on the plate of a photocopier or scanner with fingers straight.
I used a picture of my hand. We’re just going for ratios, so that should be fine, and it’s a lot easier.
Nonlinear would have been capable of producing an infinite trickle of relevant-seeming evidence
It reads to me that NL was asking for a specific amount of time to provide you with evidence, and you could have prevented this turning into unbounded delay by saying something like “you’ve asked for a week to gather evidence, and we’ll consider evidence you provide by [168hr in the future]”.
our sources would have been heavily retaliated against
Have you elaborated somewhere on this?
I understand that you’re not planning to publicly deanonymize the accounts, and I assume you don’t plan to do so privately either.
That’s right: I stopped once I was reasonably sure this approach worked. I’m not planning to do more with this. I made my scraping code open source, since scraping is something LW and the EA Forum seem fine with, but my stylometry code is not even pushed to a private repo.
I can imagine having more barriers for people to post things “anonymously” (or having them feel less safe when trying to do so) to heavily discourage some of the potentially most useful cases of anonymous accounts. … some people seem to be really worried about their reputation/potential repercussions for what they post
I think maybe we’re thinking about the risks differently? I think it’s common that people who post under an alt think that they’re keeping these identities pretty separate, and do not think someone could connect them with a few hours of playing with open source tools. And so it’s important to make it public knowledge that this approach is not very private, so people can take more thorough steps if better privacy is something they want. Keep in mind that this will only get easier: we’re not far from someone non-technical being able to ask GPT4 to write the code for this. (Possibly that would already work?)
Specifically on the “feel less safe”, if people feel safer than they actually are then they’re not in a position to make well-considered decisions around their safety.
I don’t see why people wouldn’t expect this to work on LW/EAF, given that it worked on HN?
I could have posted “here’s a thing I saw in this other community”, but my guess is people would take it less seriously, partly because they think it’s harder than it actually is. And so “here’s a thing, which didn’t end up being very hard” is much more informative.
Prediction: this works when asking humans questions too.
(The idea is, the information about the celebrity is “indexed” under the celebrity, not their parent)
~every successful callout post against a LW community organization has been built on claims of harm to vulnerable female victims.
An example of a post that wasn’t was Concerns with Intentional Insights, where most of the accusations were not about harming specific victims but for the ones that were (subverting Upwork’s minimum wage rules) the victims were mostly male.
which they would generally win
Place your bets!
It should be noted that, as I was nominally Nate’s employee, it is consistent with standard business practices for him to prevent me from talking with people who might distract me from my work; this goes to show the continuity between “cults” and “normal corporations”.
This is very much not standard business practice. Working as an employee at four different “normal corporations” over thirteen years, I have never felt any pressure from my bosses (n=5) on who to talk to outside of work. And I’ve certainly been distracted at times!
Now that I’m a manager, I similarly would never consider this, even if I did think that one of my employees was being seriously distracted. If someone wasn’t getting their work done or was otherwise not performing well, we would certainly talk about that, but who their contacts are is absolutely no business of mine.
The posts I’m referring to made claims that were much stronger than “we should be reacting more”. If you look through https://medium.com/@tomaspueyo/coronavirus-act-today-or-people-will-die-f4d3d9cd99ca and https://medium.com/@joschabach/flattening-the-curve-is-a-deadly-delusion-eea324fe9727, and the follow-up https://medium.com/@tomaspueyo/coronavirus-the-hammer-and-the-dance-be9337092b56 they’re making detailed claims about how the world is and how it will soon be.
A while ago Mitchell Porter wrote that Robin Hanson often makes arguments like:
People claim that they want to be smart and would like to be smarter. But if you stand on your head, blood pools in your brain, providing more oxygen, and thus improving your cognitive function. Yet people spend hardly any time standing on their heads. Does this mean that they don’t really want to be smarter?
Quirrel is being even more Hansonion than usual in #95, arguing that other people not trying to raise their friends from the dead means they don’t care about their friends.
We can talk about under what conditions revealing the identities of people who’ve made false accusations is appropriate, and about whether that accurately describes anything Alice and/or Chloe have done. But jumping straight to deanonymizing is seriously premature.
Speculating of course, but it reads to me like the four directors knew Altman was much better at politics and persuasion than they were. They briefly had a majority willing to kick him off, and while “Sam would have found a basilisk hack to mind-control the rest of the board” is phrased too magically to me I don’t think it’s that far off? This sort of dynamic feels familiar to me from playing games where one player is far better than the others at convincing people.
(And then because they were way outclassed in politics and persuasion they handled the aftermath of their decision poorly and Altman did an incredible job.)
Some might be willing to bite the bullet at this point, trusting some strongly held ethical principle of theirs (e.g. A, B, C, D, or E above), to the conclusion of excluding humans who lack certain cognitive capacities from moral concern. One could point out that people’s empathy and indirect considerations about human rights, societal stability and so on, will ensure that this “loophole” in such an ethical view almost certainly remains without consequences for beings with human DNA. It is a convenient Schelling point after all to care about all humans (or at least all humans outside their mother’s womb).
This is pretty much my view. You dismiss it as unacceptable and absurd, but I would be interested in more detail on why you think that.
a society in which some babies were (factory-)farmed would be totally fine as long as the people are okay with it
This definitely hits the absurdity heuristic, but I think it is fine. The problem with the Babyeaters in Three Worlds Collide is not that they eat their young but that “the alien children, though their bodies were tiny, had full-sized brains. They could talk. They protested as they were eaten, in the flickering internal lights that the aliens used to communicate.”
If I was told that some evil scientist would first operate on my brain to (temporarily) lower my IQ and cognitive abilities, and then torture me afterwards, it is not like I will be less afraid of the torture or care less about averting it!
I would. Similarly if I were going to undergo torture I would be very glad if my capacity to form long term memories would be temporarily disabled.
(Speciesism has always seemed like a straw-man to me. How could someone with a reductionist worldview think that species classification matters morally? The “why species membership really is an absurd criterion” section is completely reasonable, reasonable enough that I have trouble seeing non-religious arguments against.)
Thanks for sharing this publicly! Candid transparency is very difficult.
It’s also nice to see an organization making public something that it originally felt it couldn’t, now that some time has passed!
Smiling is communication. The information content of a message is inversely proportional to its probability. If you smile at strangers in places or situations where that’s very unusual, you’re communicating pretty strongly, and not necessarily what you intend to communicate.
For example, I live in a part of the world (Northeast US) where the culture is to leave strangers alone. You don’t talk to them, smile at them, etc; everyone just does their own thing. If someone here smiled at me out of nowhere, my first reaction would be trying to figure out where I knew this person from, and I’d probably smile back to be polite. Then if they moved on and I really didn’t know them I might wonder why they had smiled at me; were they flirting? Visiting from out of town and didn’t know the customs? On drugs? Indicating support for something I’m signaling with my appearance or attire?
I can report that nearly half of the strangers will smile back, although often they’ll look away as they do.
I’m not sure that’s as positive as it sounds? The “look away” part seems to me like it’s indicating discomfort, or that they’re worried returning your smile will escalate the interaction.
Surveyed.
The IQ question should, like with the SAT/ACT, make it clear you should leave it blank if you’ve not been tested. And the same with the follow-up in calibration.