Connor_Flexman

• Connor_FlexmanOct 23, 2021, 5:46 AM

  27 points

  in reply to: jessicata’s comment on: My ex­pe­rience at and around MIRI and CFAR (in­spired by Zoe Curzi’s writeup of ex­pe­riences at Lev­er­age)

  Regarding Eliezer’s tweets, I think the issue is that he is joking about the “never stop screaming”. He is using humor to point at a true fact, that it’s really unfortunate how unreliable neural nets are, but he’s not actually saying that if you study neural nets until you understand them then you will have a psychotic break and never stop screaming.

• Connor_FlexmanOct 20, 2021, 4:00 AM

  4 points

  in reply to: Davidmanheim’s comment on: Choice Writ­ings of Do­minic Cummings

  Not sure why you think domestic pressure /​ public agreement is strong evidence. Public pressure for all sorts of things seems hardly correlated with whether they’re beneficial.

  I think the strongest arguments for Brexit are pretty orthogonal to the economy. Things like “can the government react to crises on the order of weeks instead of months”. I do think enough crises would give us data on this but I’m not even sure it will be reasonable to extract counterfactuals from several. Other reasons to do Brexit seem similarly hard to measure compared to myopic economic impact.

• Connor_FlexmanOct 18, 2021, 10:01 PM

  4 points

  in reply to: Davidmanheim’s comment on: Choice Writ­ings of Do­minic Cummings

  FWIW, I personally don’t have much evidence to determine whether Brexit was good. Seems plausible to me that you’re right that they now just have different bureaucratic downsides. I’ve read a few things about being able to make ARIA (UK version of ARPA) and some other things from 2019 that make me lean somewhat positive, but I’m extremely agnostic. I have a bunch of thoughts on quality of evidence here, but suffice it to say I am not sure whether we will ever get much Bayesian evidence on goodness or badness. So my interest in DC is relatively orthogonal to whether Brexit turns out to be object-level good or bad (even though ideally I would know this and be able to include it in my model of how much to believe his beliefs).

• Connor_FlexmanOct 15, 2021, 9:47 PM

  6 points

  in reply to: ryan_b’s comment on: Choice Writ­ings of Do­minic Cummings

  Oh awesome, you already made the important argument here. Thanks. I’ll leave up my comment above saying similarly, though.

• Connor_FlexmanOct 15, 2021, 9:46 PM

  11 points

  in reply to: Phil Scadden’s comment on: Choice Writ­ings of Do­minic Cummings

  First, I already agreed this was true. But if you write about the urgent need for planning for biosecurity a year before a pandemic, quote a biosecurity report that mentions 8ish diseases, you cut a few from your block quote for concision, and then one of the 8 that you didn’t specific use in your block quote (but which you were definitely writing about!) occurs in a global pandemic… I just think it’s pretty reasonable to say “I wrote about this”. I might not do it per se, but if a friend of mine did it, I wouldn’t bat an eyelid. If a random acquaintance did it, I would stop for a second, think about it, decide it seemed fine. If you write a fair amount about a report warning of some things, and then one of those things happens, you get to say “you wrote about that”.

  Second, I think there’s a very important distinction between truth-seeking and truth-telling, as comes up regarding Cummings. I understand this is a pretty apologist stance but I think it’s super important here. Normally people have neither, and sometimes people have both. But I think it’s pretty consistent to have a model of him where he is truth-seeking but not always truth-telling.

  For example, he talks about reading a lot of history in a truth-seeking way and how he’s literally trying to piece together what actually happened around Bismarck because everything is so untrustworthy, and he waxes extensively on how one should interact in government in a way that continuously excises the oft-repeated political narratives and seeks truth instead. BUT he also runs campaigns that make public slogans that are slightly misleading in an unimportant way (350M), or publicly says he warned of X when instead he wrote a warning about everything in a report that included X.

  These statements are a little fishy, I agree, and you should flag him as someone who you might not want to directly believe all public statements from. It also seems fine if you go more extreme, and say he’s the type of person who you can’t trust, though others might personally still trust him and different people should have different takes on this.

  BUT I still think we should describe him as truth-seeking, especially if I flag it as “from a different perspective”. I imagine you won’t want to read much of his writing, but I think probably if you read a few posts of it you would see that he’s making a weird distinction between {how you yourself think, and how you talk to your friends and colleagues while hugging the truth} and {communicating publicly, where everybody is constantly lying, in part because you can only get across a snippet of information to people}. I don’t really know how to reconcile these personally, because I don’t know much about public communication. And I personally am not happy that he does it—in fact, it’s super annoying, because it makes it so much harder for his allies to claim moral high ground, and in fact causes a bunch of people to distrust him. But I still think pretty strongly that we should describe him as truth-seeking, because A) it’s true, and B) hardly any people in politics are truth-seeking and I think we can learn a ton from him.
  
  ETA: ryan_b makes the same point below more concisely, and provides a better example: that “the VoteLeave campaign applied basic epistemics, at his direction”. I think this is a great example, and there are other similar anecdotes, like their success in the referendum on the Northwest Regional Assembly.

• Connor_FlexmanOct 15, 2021, 9:08 PM

  7 points

  in reply to: Davidmanheim’s comment on: Choice Writ­ings of Do­minic Cummings

  Sorry, I deliberated for a while on whether to include it, but for a number of reasons decided I wanted to just ignore the politics-as-mindkiller and focus on everything else. Ideally I would have mentioned something about this, I just felt like addressing it in any respect would immediately lead to discussion about politics-as-mindkiller and not help. Also I didn’t think this post would get much publicity. Still don’t really regret it.

  I will say though, here, I think >90% of the value I got from his writings was orthogonal to ideology-level politics. I think operational-level politics is super interesting and with some effort we should be able to talk about it orthogonally to ideology-level politics, even if we are not yet at the level of being able to talk about ideology-level politics without being mind-killed.

• Connor_FlexmanOct 15, 2021, 9:02 PM

  3 points

  in reply to: ChristianKl’s comment on: Choice Writ­ings of Do­minic Cummings

  (Gove and Boris agreed in 2016 that Boris would be their push for PM, then at the last minute Gove withdrew his support and announced his own candidacy, splitting support, causing Boris to withdraw, and neither got PM. [1, 2] By a few years later, they seem to have mended things significantly.)

• Connor_FlexmanOct 14, 2021, 4:44 AM

  20 points

  in reply to: Phil Scadden’s comment on: Choice Writ­ings of Do­minic Cummings

  Many people have brought this up to me and I think it’s extremely misleading. Basically, he wrote this blog post about the dangers of possible pandemics that governments weren’t taking seriously, and heavily rested on giant block quotes from a good source, as he often does. In the block quote he included sections on like ⁴⁄₈ of the pathogens they warned about, separated by ellipses. After the pandemic he went back and added to his block quote the section on coronaviruses specifically, to show that bio-risk people were already warning about this BEFORE it happened and the government was completely failing to act on it.

  This seems like an extremely reasonable action to me—he probably should have used ETA or something, which is the only “dishonesty” I fault him for, but even that phrase is a little weird in a block quote. I can see how some people would be like “you changed it!” but absent political anger, I don’t really imagine getting mad at a friend or acquaintance for this. If I myself had a block quote that cut some things for length but was warning of essentially the exact thing that happened, I probably wouldn’t just add the section without an ETA, but I expect I would just say in an interview “I specifically warned about coronaviruses amongst other things”.

Choice Writ­ings of Do­minic Cummings

• Connor_FlexmanSep 22, 2021, 6:02 PM

  1 point

  on: Weird mod­els of coun­try de­vel­op­ment?

  Also, any recs on dev econ textbooks?

[Question] Weird mod­els of coun­try de­vel­op­ment?

• Connor_FlexmanAug 24, 2021, 11:20 PM

  7 points

  in reply to: ChristianKl’s comment on: Con­nor_Flex­man’s Shortform

  Yes, they’ve made it very clear that that’s the reasoning, and I am saying I disagree.

  A) I still think they are not correct (long evidence below)
  B) Ct values are clearly somewhat useful, and the question is how much—and I do not think the public health comms apparatus should stifle somewhat-useful medical information reaching patients or doctors just because I might be misled. That’s just way too paternalistic.

  As to why I think they’re wrong, I’ll cross-post from my fb thread against the specific pdf linked in op, though all other arguments seem isomorphic afaict. If you don’t trust my reasoning but want the reasoning of medical professionals, skip to the bottom.

  Basically, the pdf just highlights a bunch of ways that Ct values aren’t perfectly precise and reliable. It says nothing about the relative size of the error bars and the signal, and whether the error bars can drown it out—and, they can’t. To use a very exaggerated metaphor, it’s like the people saying we need to pull J&J because it’s not “perfectly safe” without at all looking at the relative cost/​benefit.

  So, they give a laundry list of factors that will produce variability in Ct values for different measurements of the same sample. But toward the end of the the doc, they proclaim that these sources of variability change the result by up to 2-3 logs, as if this is a damning argument against reporting them. The scale of Ct values is 10 logs. Hospitalized patients vary by 5 logs. That’s so much more signal than their claimed noise! So their one real critique falls very flat.

  However, they do understate the noise significantly, so we can strengthen their argument. Within-patient variability is already like 2-3 logs, as you can see from data here for example: https://​​www.ncbi.nlm.nih.gov/​​pmc/​​articles/​​PMC7151491/​​. So variability of viral loads across different patients, different collection methods, and different analysis methods will have more like 4-6 logs of variation. That’s the stronger argument.

  But even this is ultimately too weak.

  Most of the variation is on the negative side: there are lots more ways to fail to get a good sample than there are to accidentally find the virus is more concentrated than in reality. So, low Ct values indicating high viral load are still very good signals! I don’t know the exact numbers here because they won’t report them many places, but your reasoning would hypothetically go: If you get a Ct value of under 20, you better start canceling meetings and preparing for a possible hospital visit. If you get a Ct value of 38, maybe it’ll end up getting much worse, or maybe not. Not much information there. This is simple reasoning—doctors do it all the time with other tests with high falsity rates, saying “if you test positive on this you probably have X, but getting a negative doesn’t rule it out.”

  And aside from this asymmetry, just the correlation is also really useful! I am not the first person to say this: googling turns up a bunch of instances of medical professionals saying similar things:

  • https://​​www.sciencemag.org/​​news/​​2020/​​09/​​one-number-could-help-reveal-how-infectious-covid-19-patient-should-test-results

  • https://​​www.aacc.org/​​science-and-research/​​covid-19-resources/​​statements-on-covid-19-testing/​​aacc-recommendation-for-reporting-sars-cov-2-cycle-threshold-ct-values

  • https://​​www.aacc.org/​​cln/​​cln-stat/​​2020/​​december/​​3/​​sars-cov-2-cycle-threshold-a-metric-that-matters-or-not

  • https://​​directorsblog.nih.gov/​​tag/​​ct-value/​​

• Connor_FlexmanAug 23, 2021, 9:25 PM

  7 points

  on: Con­nor_Flex­man’s Shortform

  Another sad regulation-induced (and likely public health comms-reinforced) inadequacy: we don’t report Ct values on PCR tests. Ct value stands for cycle threshold, which means how many cycles a PCR amp has to do before the virus is detected. So, it directly measures viral load. But it isn’t reported to us on tests for some reason: here’s an example document saying why it shouldn’t be reported to patients or used to help them forecast progression. Imo a very bad and unnecessary decision.

• Connor_FlexmanAug 23, 2021, 9:04 PM

  5 points

  in reply to: Connor_Flexman’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  Another cool data point! I found a paper from Singapore, Jul 2020, testing tear swabs but incidentally giving a bunch of PCR tests too. I’m much more likely to trust a paper that gives PCR tests incidentally, rather than is directly testing their effectiveness with researcher bias toward better results. By counting up the squares by hand, this paper shows ²⁴⁄₁₀₈ PCR tests came back negative if I counted correctly: that’s 22% false negative rate (FNR).

  Now, for adjustments:

  • First, these patients were recruited from a hospital. So they obviously have much higher viral load than the average person, so we’d expect higher FNR for the general population. (And we see the expected relationship between viral load and positive results: people with average low Ct values (meaning high viral load) rarely test negative, but those testing negative lots have very high Ct on their positive tests.)

  • On the other hand, only ²⁄₁₇ patients test negative >50% of the time; a lot of the negatives come near the end of a patient’s sickness or hospital stay. So we don’t see great empirical evidence for the hypothesis that some people are consistent false-negatives. If you take out the negatives-at-the-end effect, there are far fewer false negatives, maybe 5-10%. However, this is basically moot because of the selection effect for the hospitalized as mentioned above. Of course you’ll see hardly any consistent-false-negative-patients in the hospitalized!—the fact you see any macroscopic number of false negatives in the middle of progression is a terrible sign (and, if there were any fully-false-negative patients, we wouldn’t see them anyways! Bad filter).

  • And we do see the requisite theoretical evidence. Because of the two patients with repeated false negatives and low viral load when positive, we can easily extrapolate that some patients just have slightly lower viral load and test negative consistently.

  Overall, there isn’t much easy way to convert this study into “FNR on asymptomatic individuals who get tested”. However, I think if 5-10% of tests on the hospitalized came back negative, that strongly implies more than a 20% FNR on the asymptomatic. I would personally guess that this lends credence toward 10-40% FNR on the symptomatic and 20-80% FNR on asymptomatic. (Lest I double-count evidence, let it be known I’m basing these numbers in part on the above analysis of my personally-known symptomatic individuals with ~40% FNR.)

  What links here?

  • Connor_Flexman's comment on Home Anti­gen Tests Aren’t Use­ful For Covid Screening by Elizabeth (May 8, 2022, 11:23 AM; 5 points)

• Connor_FlexmanAug 7, 2021, 9:30 PM

  4 points

  in reply to: Ethan Perez’s comment on: ($1000 bounty) How effec­tive are marginal vac­cine doses against the covid delta var­i­ant?

  I have a tentative answer! Some cursory googling makes me think that J&J also just replicates the spike protein in you, the same way Pfizer/​Moderna do. This means it’s just strictly less effective. Then you’d want to just do the Pfizer/​Moderna one that you haven’t yet—unless Elizabeth’s comment about limited mRNA vaccine doses is decision-relevant, which I still haven’t looked into.

• Connor_FlexmanAug 6, 2021, 12:40 AM

  15 points

  in reply to: James Payor’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  In short, I don’t really know how it can be as bad as I claim it is. It seems like it should straightforwardly be highly accurate because of your two points: the sensitivity should be at a much lower threshold than the amount needed to infect someone.

  Yet, I still believe this. Part of this belief is predicated on the heterogeneous results from studies, which make me think that “default” conditions lead to lots of false negatives and later studies showed much lower false negatives because they adjusted conditions to be more sanitary and less realistic. However, this is just an extrapolation, and I haven’t looked into these studies unfortunately.

  The bigger reason for my belief is that I’ve seen several people almost-definitely get COVID and then test negative.

  1. First data point: B was out in public unmasked, got it, then their family got it. 4ish people showed symptoms, one didn’t. 2-3 tested positive for COVID, the others didn’t, including 2 who tested negative 3ish times in a row, using PCR. B was one that tested negative repeatedly. B was notified shortly afterward that the person they were with in public had tested positive for COVID.

  2. Second data point: C went back to university in Sep 2020. Two of their family members visited. Shortly after, C got pretty sick and tested positive for COVID. Then their family members got pretty sick with flu-like symptoms. Both family members went to the doctor after symptoms and tested negative by PCR.

  3. Less-strong data point: D flew on a plane from the Bay late-Feb/​early-Mar 2020. They landed, two days later they got sick with cough, maybe more, felt pretty bad, and had an spO2 of 85. They tested negative 2-3 times by PCR.

  I heard about these cases because they were fairly close to me. There were maybe 2 other cases as close to me as these, so these represent about half my epistemic exposure to COVID cases on the ground.

  I don’t know how to possibly parse the first two cases aside from saying that the PCR tests gave false negatives. You can’t even say “they got the flu”—their family members tested positive for COVID! The best alternative explanations seem truly terrible: there’s a minuscule chance I just got a wildly skewed sample, or I could’ve done a truly abysmal job at noticing some selection effect. So I feel like I basically have to take them at face value. Using data points 1 and 2 only, and adding 2 positives from cases close ot me, the PCR tests gave false negatives roughly ⁷⁄₁₀ + ²⁄₃ + 0/​2= ⁹⁄₁₅ times, ~60% false negative rate, but if you count by person this is only ~40% false negative, which is the better way to look at it to correct for the selection effect of people who test negative getting tested repeatedly. Maybe someone got rapid when they thought they got PCR, and you lower 10% to bring the total to 30% false negative rate by person. But not much more in sample mean.

  As I said, I don’t really understand how PCR tests can be this bad. However, it would tie up very neatly if, for example, COVID just didn’t make it to the nose in a lot of patients. Perhaps orders of magnitude more is coughed out of the lungs as an infection vector than is exuded from the nostrils. Or perhaps the efficacy of swabbing varies a ton, or the efficacy of testing—a lot of the negative tests I know about were from Red states, and I can’t help but wonder if the old “getting the results you want to get” effect is striking in another wild circumstance (but of course note the selection bias since I know of more cases in Red states).

  And even without knowing how PCR tests can be so bad, I don’t feel like I’m going that much out on a limb when I’m imagining there might be lots of heterogeneity in how they’re done. Even if the best tests are really quite good, if the worse tests have user-error rates of one in five, and these are selected to be the ones more in use where COVID outbreaks are (due to culture or the obvious causality), you could potentially have a lot of people with 20% FNR rates. Also, while I think lots of criticisms that “the lab is different than the real world” are misplaced, COVID tests seem like almost a central case where you’d expect that specific failure mode.

  (I probably won’t delve into the papers to try to figure this out, but I would love to hear from anyone else who might have alternative hypotheses about this, or reasons why “COVID doesn’t always go to the nose” shouldn’t be the default hypothesis here.)

  What links here?

  • Connor_Flexman's comment on Home Anti­gen Tests Aren’t Use­ful For Covid Screening by Elizabeth (8 May 2022 11:23 UTC; 5 points)

• Connor_Flexman 5 Aug 2021 23:32 UTC

  1 point

  in reply to: Sounder’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  E.g.

• Connor_Flexman 4 Aug 2021 21:32 UTC

  3 points

  in reply to: Owain_Evans’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  Oops, thought that was a top-level reply to me when I clicked on it, rather than a reply to Adam. Sorry. Makes more sense in context.

• Connor_Flexman 4 Aug 2021 20:54 UTC

  3 points

  in reply to: Owain_Evans’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  Great paper, thank you!

  (Do you mean the Lancet /​ British intelligence test paper when you say ONS? I embarrassingly don’t see a paper I cited with those letters in it.)

  The current way I imagine citing this is to use as a corroboration of my rough estimate of <2% 30yos having Long COVID. I don’t see an easy way to integrate it with IQ loss estimates—since I wouldn’t expect tiny levels of IQ loss to show up on a survey about actual Long COVID symptoms, it seems relatively consistent for 2% symptoms after 12 weeks to still correspond to an average IQ loss of .15 points after 12 weeks (~10% lose 1 IQ point, 1% lose several IQ points). I do think it points downwards somewhat, though, maybe a factor of 2?

• Connor_Flexman 4 Aug 2021 20:21 UTC

  22 points

  in reply to: AdamGleave’s comment on: Delta Strain: Fact Dump and Some Policy Takeaways

  I should probably argue with Matt directly, but my brief take is that this is just entirely incompatible with what we see on the ground. The friends of mine who got COVID aren’t reporting 45% chance of their life being 20% worse. That’s… an incredibly massive effect that we would definitely see. Would anyone realistically bet on that?