Florin’s right that the 15-64 age group doesn’t pain a clean picture of the actual numbers since it combines very different excess death rates, but even the 25-44 group experienced a serious increase. Rather than Katja being “wrong,” they are very much right.”For the next 2 years, you will have a 25% higher risk of death than usual” is not a high absolute risk of death, but that shift from baseline is not just “not entirely insignificant” either.
Right, there is a ton of misunderstanding regression floating around on this issue it seems. Yet, one would still think that Having covid would be more predictive of Long covid than Believing you’ve had covid, since Believing and Long ought to be correlated only through their shared association with Having (common cause rather than mediation). The fact that this is not the case could indicate that people with chronic conditions come to think they Had covid (discussed at the end of the study) or that the measure of Having covid is not that good (see Siebe’s comment), or that it’s psychosomatic (loose usage of the term), or something(s) else.Adding to the uncertainty is that “less than half of those with a positive serology test reported having experienced the disease.” This is especially troublesome since participants were informed of their serology results prior to the self-reports, so that’s some weird denial (or misunderstanding). However, that doesn’t mean they are unassociated! They are associated! (Table 2) Only 2% of seronegatives believed they had covid, but 42% of seropositives did. Having-->Believing, so that’s good at least.
My worry was that maybe an antigen throat test would need a different design/reagents/whatever (since there’d be a lot more saliva, etc.) than an antigen nasal test to be sensitive. Apparently the health authorities will not explain any of the “under the hood” issues (just that a throat swab is more difficult, and therefore more dangerous, to do to yourself), and the expert WaPo got is worried not about false negatives but false positives! First, the specificity of the tests are great so it’s hard to fathom what would be introduced to drop that, and second, false positives (as long as they aren’t extremely common) are fine if we’re talking about spread control. Plus, a quick Googling on flu nasal/throat swabs kinda suggests there’s probably not a design issue. Throat swab it is!
A single omicron antigen test is good after showing symptoms, not before (Table 1). The good news is antigen doesn’t miss randomly (of course not) - it misses lower viral load cases (Figure 1a). But we can’t be sure those cases are at a steady-state of viral load, so it doesn’t necessarily ensure that missed cases remain low viral load. Asymptomatic cases tend to be lower viral load than symptomatic cases (makes sense), but it’s by no means a guarantee (Figure 1b-c, median Ct of symptomatic is ~25 while median Ct of asymptomatic is ~30, higher Ct means lower viral load).https://www.medrxiv.org/content/10.1101/2022.01.08.22268954v2.full.pdf+html
Yeah the “ethical rules” linked tweet asks, since tests are available in the UK, what if we just had Londoners take two—one in the nose and one in the throat, to see if they work? (so a non-confident version of #3)It’s more complicated too, not just #2 of developing tests we know work with saliva. From the linked preprint, the viral loads are somewhat higher in saliva than nasal earlier but nasal than saliva later (low sample size for this inference though).And those data are a bit sad as they show that regardless of the saliva/nasal viral loads, antigen struggles even for high viral loads until day 3 post-PCR-positive. PCR can detect pre-symptomatic cases (so day 0 PCR-positive can be...fudging a bit for the shorter serial interval, maybe day −3 of symptoms for omicron), which implies that taking an antigen test when you develop symptoms is right on day 3 post-PCR-positive. Maybe the FDA was getting signals of this (additional citations also included in the linked preprint) when they issued their cryptic statement about lower antigen sensitivity for omicron. It feels like we had more wiggle room on an informative antigen testing window for delta than we do for omicron. Antigen testing was weakly informative pre-symptoms for delta, but, based on this preprint, it seems antigen testing is wholly uninformative pre-symptoms for omicron.
And we have VAERS, to which individuals can report directly. Plus, the surveillance system (including our crappy contact tracing systems run by the states) means we get sub-hospitalization data. Ideally contact tracing would also help arrest spread (not so much if they call you 3 days after you test positive 3 days after you first show symptoms...sheesh), but at the very least you’re getting a survey done.I think just from becoming aware of the surveillance and adverse event reporting systems, Valentine’s base for a high degree of skepticism is pretty shaky. Being armed with an understanding that actually, the mechanisms by which the data could be generated DO exist should help a lot. I want to note that when people exclaim we should trust the experts, I believe it is about this level of ignorance they rightly have in mind (props for identifying a knowledge gap and honestly seeking to address it!) - lacking key fundamental knowledge necessary to even begin to assess the veracity of claims, rely on the people who do have it! As we learned from the pandemic fiasco though, our “experts” having the ability to generate and interpret that information does not mean that they always do it well. I’ll also say that even without ongoing adverse event monitoring or observational effectiveness studies, the clinical trials were gigantic and provide strong evidence supporting efficacy and safety. [Unless the researchers were selecting what data to collect in which case seeing the raw “data” would be meaningless too. Sadly, data tampering or fabrication happen, but if that fact will undermine your reliance on any data generated by anyone but you, frankly there’s no convincing you with other people’s data.] I’m not sure how large any selection biases are, but I imagine they would have to be huge to impeach such extensive data vs. a handful of anecdotes that are themselves not free of selection either.
You know what we still never got anywhere on settling, and which is super relevant right about now? The extent to which vaccines make some people immune to infection while others largely aren’t, versus the extent to which they make most people less vulnerable to infection in each encounter but not fully immune....My model now says it’s a hybrid. People have different levels of antibody and other responses to the vaccines, which means some people are effectively fully immune (at least for a while), others get more limited protections
You know what we still never got anywhere on settling, and which is super relevant right about now? The extent to which vaccines make some people immune to infection while others largely aren’t, versus the extent to which they make most people less vulnerable to infection in each encounter but not fully immune....
My model now says it’s a hybrid. People have different levels of antibody and other responses to the vaccines, which means some people are effectively fully immune (at least for a while), others get more limited protections
This is definitely an important question but it doesn’t seem to me as so wide open a question. I think the prior (i.e., established POV of “Science”) is just your model, and I think the evidence is consistent with that. The natures of the humoral and cell-mediated immune systems would seem to suggest that if you have few or low-capability antibodies/cell instructions, you’d be more likely to get sick, and that at a critical mass, you’re functionally immune because anything that enters will get dealt with (e.g., think border patrol). This would look like a sigmoid curve relating neutralization titers to vaccine efficacy.
If you take a snapshot of neutralizing titers on average induced by a vaccine vs. efficacy, then you could naturally wonder if this is just because of a heterogeneous immune response in the population (and/or sampling over time, e.g., although the average time since dose may be X, some samples may have been taken at X+30 days and therefore have lower levels from natural clearance over time) or if it’s simply probabilistic. Once you see titers over time and/or for multiple vaccines, or at the individual level for multiple people/samples, you can test whether that variability matters, and you can find that indeed immune responses are heterogeneous (not just binary though), resulting in functional immunity around a threshold titer level and probabilistic immunity below that (and as commonly assessed, those categorized with serological non-responsiveness below a lower threshold). Not surprisingly, the titer vs. efficacy relationship is sigmoid.
In fact, neutralization titers represent the amount you can dilute a sample while still preventing infection in 50% of inoculations (the more you can water it down while to get it down to 50% “efficacy,” the more antibodies there must be), so there is a probabilistic component even at that point (or at least error terms to our model), but certainly “the more [antibodies], the merrier [the human],” and at some point you have so many antibodies floating around (or your body can whip them up on a dime, for other diseases) that you’re functionally immune.The case isn’t closed, but your model is the prior, the prior is your model. Woo!
I’d be hesitant to conclude from prices -naturally- skyrocketing that welfare is lower. “Reasoning from a price change” as Scott Sumner would say. If you have a shortage due to supply constraints, and innovation eases the supply constraint and unlocks complementarity value in other products, that’ll be reflected in their prices and does not necessarily mean people are worse off.I like your positioning of Braess’s paradox as an externality. It’s a special case in that it isn’t the participation in the system that exerts a social cost but the particular pathway of participation that does. I suppose because the traditional economic analysis assumes homogeneity in lots of dimensions of the problem (for the most obvious example, there aren’t multiple pathways to participate in the market, just one—a non-descript exchange at a strike price) that it would be challenging to characterize a multi-path system like this as a typical economic market.Perhaps as you mention with innovation, we could approach this from one step up from the market at resource-based and production-function-based generation of supply. Although we might say there is one non-descript type of “exchange” that is participating in the market, before you get to that node, there are other nodes you could take, which facilitates modeling this as having multiple pathways. Consider the decision to use greenhouse gas scrubbers in production or not (assume no regulations). For most companies, “not” is the dominant option to maximize profit, which constitutes the vast majority of their utility. For other companies choosing to maximize their own slightly different utility function, using scrubbers is desirable. Then one introduces a new scrubber that is way cheaper and becomes adopted by some marginal firms (reducing the total externality, reduction on net) who then can increase their production to get their total emissions back to the previous aggregate level they were comfortable with but at their lower emission/unit level (increasing the total externality, no change on net), becomes adopted by prior “green” firms (reducing the total externality, reduction on net) who can increase their production to profit a bit more while keeping some of the environmental gain intact (increasing the total externality, slight reduction on net), and prompts entry by marginal would-be firms (increasing the total externality, the net depends on the sizes of all the margins). What if the short-term net effect is helpful, but if down the line this leads to the old scrubber producer exiting, demand overwhelming the new scrubber producer, price increases leading to insufficient adoption or even to disadoption, and ultimately the total externality sneaking back up and over where it was originally?As with most paradoxes, they are largely a function of an ill-defined problem, an analysis at a “different level” than needed, or neglecting relevant complexities. In this case, it’s specifically failure to consider all the margins.Another example could be in the supply chain itself, which is great for the overloading aspect. Your firm promises 30 day delivery because your models say your company could do 28 days at the current level of demand. You innovate that down to 25 days and start promising 28 day delivery (you even net a day of safety, right?!), but new demand overloads your node (or even one specific upstream node that is now part of your innovative process), and you can’t even deliver in 30 days now (maybe this is really, thanks Wikipedia for the name to it, Jevons paradox, and perhaps so is my emissions example, but I think either way the point is that there is a “missing margin” that planners didn’t see, leading to overload from an “improvement”).
You may also enjoy Why the West Rules—For Now, which also addresses environmental, rather than institutional, factors. As Kaj_Sotala notes, these kinds of books are often entertaining reads but just-so stories.
This is again a threshold, not comparator, complaint. Ct values are generated by PCR. Instead of using a crosstab for all samples, this approach is to use a crosstab for a subset of samples with higher viral load. It’s reasonable! IIRC from a previous paper, this (90% of Ct<25) has a similar effect as just reducing the overall cutoff to (80% of all). It’s also reasonable to use studies from other countries or to follow other agencies, in either case the ones we think are credible, which is again about the evidence threshold. What I’ve been hammering on is that the idea these tests are so different that they’re noncomparable is not sensible.
I somewhat like the distinction between “testing for infectiousness” and “testing for whether I have it” (especially from a public health, rather than personal healthcare, standpoint). “People want to go to parties so they want fast, even if slightly less sensitive, tests because sometimes they don’t really even care about their own health status, just whether they can reasonably party” is also a great reason to try to market the product (let party organizers or other organization police what tests they will accept or whether they will expect pre-testing), but the FDA needs to assess that sensitivity somehow. What is the standard for infectiousness that the FDA is going to use to have a second evaluation pathway if there really is a distinction with a difference here? Probably some indicator of the amount of infectious virus present. A plaque assay (yields plaque-forming units, PFU) would be great, counting infectious particles, but the test itself takes days, needs BSL-3 conditions, and is resource intensive in other ways as well. Don’t compare to PCR, compare to PFU...that’s even more demanding against would-be antigen test manufacturers. And PFU has a pretty strong relationship with Ct. Compare to PCR then!People are rightly dissatisfied with the status quo and looking for someone to blame for the lack of tests, but the problem is not that the FDA compares to PCR, it’s the threshold they wanted. I know “evidence-based decision-making” is abused and gets a rap, but I don’t think effectively advocating for evidence-less decision-making will make us less wrong (in case that sounds too harsh, what was the alternative to PCR that was supposed to validate antigen testing for us?).
The requirement for products to have the same cost/benefit profile really hampers innovation in the marketplace. A less sensitive test (literally, as a % of PCR) over the cumulative test-testing window (e.g., −2 to +5 days from symptom onset) may be desirable when used in a specific part of that window where it doesn’t actually have as severe of sensitivity disadvantage (e.g., −1 to +1 days). Depending on the disease, we may not want to compromise on specificity at all. These are just the “cost” profiles (haha I left out price) - the personal benefit is diagnosis (it’s the same whether it’s PCR or antigen).But there is also a public health benefit to early diagnosis, which does differ across the tests (depending on test provider and logistical situation, this could be a 1 day vs. 15 minutes difference, or several(!) days vs. 15 minutes difference). Particularly in a situation where we’re trying to stop the spread of a disease that doesn’t need immediate serious treatment, a higher false positive rate may be acceptable (note: that means lower specificity, not lower sensitivity). This is why it makes sense to let a variety of products on the marketplace that meet a minimum threshold so the world can tradeoff along these various attributes (the FDA is charged with finding an appropriate quality floor, but that’s just a matter of dialing in the floor cutoff). Whether these tests will be prioritized by public health authorities (and the public) depends on the actual public health policy smorgasbord adopted and tastes among test properties.So the FDA, inspecting these cost and benefit profiles, is not approving a test for diagnostic use based on its public health properties (which depend on the whole public health approach) and is instead looking at its personal diagnostic properties. Is the test not actually for diagnosis? That should certainly change the criteria. And I think, clearly, antigen tests are for diagnostic use. The goalposts of antigen vs. PCR testing are not different—“they test for different things” is a mischaracterization of what they do. They use different indicators but are testing for the same thing (COVID) to inform the same behaviors (which is also evident in how we’re talking about them as a substitute for PCR!). Notice that antibody testing uses yet a different indicator to test for a different thing (adaptive immune response) with different behavioral goals.You may think the quality floor was set too high [I think so too, if that would have meant 80% sensitive tests in 2020 giving way to the 90% ones we see now, given the imperative of testing capacity] (or that quality floors shouldn’t exist at all, which is very delenda est) and generally the FDA has been too strict and slow in this emergency, but that’s not the same complaint as saying the FDA has been unfair or stupid on antigen tests in comparing to PCR.
They’d be vaccinated-lite. The neutralization titers in vaccinated plasma are better than in convalescent plasma. Lots of room to complicate things and get it closer to reality, but that doesn’t touch the public good value thing so much.
Trevor Bedford gets into this, and the short answer is technically yes, but the important part about decomposing Rt is not the decomposing it per se but the info it yields on how much vaccine escape might be going on. For reasonable R0s, there has to be substantial vaccine escape. https://twitter.com/trvrb/status/1466076797670363140
Regarding translating fold reductions of neutralization titers into vaccine effectiveness, I always go back to this: https://www.nature.com/articles/s41591-021-01377-8 (Fig 1a). Titers ~4x convalescent is what mRNAs do against wild-type, getting them to about 90%+ efficacy. About a 2x reduction against Delta means titers ~2x convalescent, getting them to about 85% efficacy. These conform with what we’ve seen. A 25-40x reduction against Omicron means titers .10-.16x convalescent, getting us down to 40% efficacy for two shots. Pfizer says a booster gets us back to “full” efficacy, and hopefully that holds up!
Good job looking for cruxes! I agree with you that quantifying a differential in exposures would help nail down how much we should favor vaccination (or not), but the idea behind the probabilities I laid out was getting at the risk of inducing asymptomatic-spread. At the most unfavorable to vaccination (like how I also assumed vaccination leads to only asymptomatic disease), asymptomatics generate N infections from N exposures with p=1 and symptomatics generate exposures with p=0 (because they quarantine), so we can just look at the risk of inducing asymptomatic-spread without additional layers of calculation.Though that does indeed depend on the key probabilities going into calculating the risk. If p>>5%, then additional calculation would be warranted, and calibrating the probabilities better would be more important. For example, it’s clear just looking at the conditional probabilities that the turning point is when the relative risk of infection depending on vaccination equals the reciprocal of the relative risk of being asymptomatic conditional on infection depending on vaccination—that is, if vaccinateds are twice as likely to be asymptomatic conditional on infection than unvaccinateds (wow, the RR is a little under 2, but let’s call it 2, Fig 3), we prefer vaccination as long as vaccination cuts the risk of infection by at least half (vaccine effectiveness >= 50%). Any less than half, and then we can’t just prefer vaccination out of hand and have to go through and calculate. And then figuring out the actual differential in exposures (and viral loads!) would be relevant too.I agree with you that the probabilities I’m focusing on are in a much narrower time frame and that widening it out, p will lift off from the rate estimated in the clinical trials (about a 2 month window). As the vaccine effectiveness rate approaches 0%, then indeed we can’t prefer vaccination out of hand. How would that happen? As you suggest, with a long enough time window, the attack rates could equalize at 100%. I don’t actually see that happening (I expect the vaccines don’t only provide probabilistic protection of around 85% but, at least for some, effective immunity). But vaccine effectiveness could reach the reciprocal of the relative risk of being asymptomatic conditional on infection depending on vaccination well before getting to 0%. If you think vaccine effectiveness for the long term will fall below 50%, then we have some more calculating to do. Seeing as effectiveness has stayed about as high as models would tell you, falling below 50% only seems like a real possibility with Omicron, and my guess is we’ll either get a new shot to avoid lower effectiveness   or learn that 3 doses work against it.My prior on vaccine effectiveness staying over 50% even in the long term is strong enough, and the extra research and calculation that would otherwise be required to address this further is daunting enough, that I’ll leave it at that. I don’t want to say the burden of proof is on either of us here, since ultimately it depends on which prior is “deemed” the prior.I want to reiterate that your general point that a vaccine might not have the public good value we assume it has is legit. We are used to diseases that generate symptomatic infections with high p, so any reduction in symptomatic infection is noticeable and contributes to stopping the spread. If a vaccine pushes infections to “hide” in asymptomatic ones instead (because the disease generates symptomatic infections with low-moderate p), and asymptomatic infections are still highly transmissible, the public good value is not quite so certain, generally speaking.
Your POV really turns on (emphasis added):
Having a relatively rare belief that vaccinated people seem much more likely to get asymptomatically infected and to have lower mortality BUT also noting that vaccines do NOT prevent infectiousness and probably cannot push R0 below 1.0.
Much more likely than what? It would seem the relative comparison you want to make would be vs. the unvaccinated, but that’s obviously false (and that’s the important part). It’s true they are more likely to be asymptomatically vs. symptomatically infected (yay mild COVID), but so what? Most of the work is done on any infection at all, e.g. (making up numbers but illustrating the point):P(infected | unvaccinated) = .50, P(asymptomatic | infected for unvaccinated) = .50, and then assume that symptomatic people are less likely to transmit the disease than asymptomatic people because they know to quarantine thanks to the symptoms. So that’s a 25% chance of getting asymptomatically infected feeding into a decision generating a negative externality.P(infected | vaccinated) = .05, P(asymptomatic | infected for vaccinated) = 1.0, let’s assume there are no symptomatic cases of infections among the vaccinated (hahaha), that’s a 5% chance of getting asymptomatically infected feeding in. Again, most of the work is done on any infection at all, so having a higher chance of symptomatic (vs. asymptomatic) infection doesn’t really matter (at the level of vaccine effectiveness and rate of asymptomatic infection we’ve seen).
Do vaccines prevent infectiousness? I remember seeing CDC data over the summer about how symptomatic vaccinateds are as infectious (in viral load) as symptomatic unvaccinateds, so that’s conditional on showing symptoms. But let’s assume asymptomatics in each group are also equally infectious—then we can still favor vaccines because, see above, most of the work is done on any infection at all.
To conclude, I think it’s extremely clear that your (2) is wrong. There is public good value to vaccination.
I like the distinction between target-only forecasting and reference class forecasting.It’s interesting you use the mathematical terms zeroth and first order (and higher order) approximations, when one could take reference class forecasting into statistical terms instead:1. Identify a reference class (relevant population from which the target was drawn)2. Model it and make predictionsThe zeroth order approximation is Yi=b0. Intercept-only model, your prediction is the average from the reference class.The first order approximation is Yi=b0+b1*X. Now there’s a slope, your prediction reflects the fact that a characteristic of the observations in the reference class varies and you can improve your prediction from that.Higher order approximations...maybe such a simple model isn’t the way to go (log-transformation? other predictors? interactions? non-linear sigmoid?).The imperative is to try to understand the mechanism(s) that generate the reference class from which your target will be/is drawn (model it!) to make good predictions about that target.