What Alex Tabarrok called The Paxlovid Paradox is getting noticed by the people who notice such things, and completely ignored by everyone else. I’ve split off this week’s Paxlovid update to help make the situation easier to notice, and easier to remember and reference later, lest we forget.
Supply and Demand
The good news is that we are confident Paxlovid is safe and effective and the purchasing department is acting accordingly. Once we decide it is legal, we’ve secured our supply. The Biden administration has agreed to pay $5 billion for 10 million treatments. I previously thought this was 10 million pills as did the source, but I’ve been informed it was 10 million treatments, which means I was about to be happy to pay ten times what we’re actually paying. Kind of neat.
They’re going to make a profit from saving people’s lives. Outrageous!
The bad news, of course, is that Paxlovid Remains Illegal.
The trial was stopped due to ‘ethical considerations’ for being too effective. You see, we live in a world in which:
It is illegal to give this drug to any patients, because it hasn’t been proven safe and effective.
It is illegal to continue a trial to study the drug, because it has been proven so safe and effective that it isn’t ethical to not give the drug to half the patients.
Who, if they weren’t in the study, couldn’t get the drug at all, because it is illegal due to not being proven safe and effective yet.
So now no one gets added to the trial so those who would have been definitely don’t get Paxlovid, and are several times more likely to die.
But our treatment of them is now ‘ethical.’
For the rest of time we will now hear about how it was only seven deaths and we can’t be sure Paxlovid works or how well it works, and I expect to spend hours arguing over exactly how much it works.
For the rest of time people will argue the study wasn’t big enough so we don’t know the Paxlovid is safe.
Those arguments will then be used both by people arguing to not take Paxlovid, and people who want to require other interventions because of these concerns.
FDA Delenda Est.
Scott Alexander notes that the current prediction market median approval date is January 1, 2022, with a 92% chance of approval by March. I’ve asked Polymarket to put up a market I would trust somewhat more, which should be up in a few days. My guess is that this the current median estimate is somewhat too pessimistic.
He goes through and quickly destroys many plausible reasons why this delay would make sense. In the comments several attempts were made to raise additional objections, with concerns about the ‘manufacturing process’ being the closest to kind of maybe being able to pretend to be a real consideration (as opposed to the others, which were at best pretending to pretend). I still don’t think it does an especially good job of pretending to be a big enough concern to matter, and was disappointed by Scott’s respect for the objection, which I’ll discuss more in detail below. Whatever pills we have now were somehow manufactured, without expected delays we would have made a lot more of them a lot faster, and we could have dealt with that issue while doing the trial if it was the true bottleneck and the incentives had been reasonable.
This is noteworthy, not because I think it’s identifying the causal mechanism (I think it mostly isn’t) but because it describes a world that makes more sense and stopping the trial unethical but a reasonable response to incentives. If only this were about people who responded so well to pure dollar incentives our problems would be so much easier to fix.
It is not hyperbole, but it is imprecise. What’s our best guess of how many Americans will die as a result of the FDA’s inaction on Paxlovid?
For a given person, Paxlovid saves their life if and only if:
They get Covid-19 and would have died without Paxlovid.
Covid-19 gets detected early enough to use Paxlovid.
They are given Paxlovid.
They live.
We would need to subtract any opportunity costs or side effects from Paxlovid, but for now it seems reasonable to assume the side effects are small when used on Covid-19 positive patients when compared to the benefits, and that the opportunity costs are also small since Paxlovid should combine well with other known-to-be-effective options.
We need to estimate the number of people who are going to die between now and approval in order to do step one. My guess is that 50,000 is somewhat high, since deaths lag and the surge is only starting now, so if we are starting now I’d use more like 40,000. But it’s better to start at least two weeks ago if we’re computing the overall death toll, so it’s more like 60,000 for those purposes. For now, we’re going to assume that ‘as soon as the clinical trial data is clear as day plus enough time to call and have a meeting’ is a reasonable starting point.
Step four only works about 90% of the time, so we need to multiply the number of dead people by 90%, as the other 10% couldn’t have been saved. That’s the other easy one.
Then we need to figure out how many of those who die detect Covid-19 fast enough to get Paxlovid when it would be effective. That’s trickier, especially with our current lack of tests, although that’s also the FDA’s fault. The failure rate that matters is that among cases that would be fatal without Paxlovid, so there’s biases pointing both ways—cases that are are harder to detect are mostly harmless, but cases where the patient doesn’t seek treatment fast enough tend to be deadlier. Given that our best guess is that most cases are never detected, the first consideration dominates. Also the existence of Paxlovid would make people more inclined to get tested. What is our best guess as to the Paxlovid-world’s detection rate? I’m going to guess this is something like 90% as well, the window is reasonably wide, but I’d also believe substantially lower, so I’ll use 80% to cover model uncertainty.
That leaves the big question. If the FDA approved Paxlovid, how many doses would be available? Would they be distributed to the people who needed them? Would those doses represent doses that would not otherwise be used, or would they move consumption forward that would have happened anyway?
This is trickier, and I don’t have the answer if we look only using causal decision theory on this particular choice. Paxlovid is 90% effective but the effectiveness per pill is many orders of magnitude lower since most people would recover anyway, and there were only 7 deaths in the entire clinical trial control group despite them enrolling over 1,000 patients.
It’s entirely possible that, while making Paxlovid widely available today instead of widely available on January 1 would save about 30,000 lives, given the delays already built into the system, the delay on approval won’t on net cost many lives. If it’s true that Pfizer will have about 180,000 treatments by end of year (Tweet was corrected), and each patient takes ten pills, then we could only treat 180,000 patients, and that probably saves about 5,000 lives (depends on how selective you think we can be in finding a vulnerable group, we can do better than the study but earlier my estimate of this was clearly too high and I was thinking 20k), less if the bottleneck would have continued for a while, but given how close it is the bottleneck presumably wouldn’t last too much longer. Which is only a moderate decrease.
This is relevant to the value of doing an end run of sorts around this particular decision point, or leaning on the FDA regarding it, if you don’t count impacts on future other decisions.
Responding to Incentives
That still treats the manufacturing process as if it couldn’t have gone any faster, or at least that the decisions made in how fast to go weren’t interlinked with expected future FDA behavior, at least in the past. If Pfizer would have expected to FDA to approve Paxlovid a month sooner, that’s a strong reason to get more pills made faster, even when there isn’t yet full confidence that the drug is going to work. Similarly, an advance commitment to paying a higher price for the first however many pills delivered quickly, conditional on the drug working, and ideally to also cover some costs if it failed, would have done a similar thing.
It’s a magician’s trick. The FDA causes the delays, and not in a strange functional decision theory or acausal trade kind of way. They do it in the direct obvious kind of way. It’s worth quoting the rest of Eliezer’s thread.
If one is counting the invisible graveyard now that it is about to both grow far larger and be much more highly visible, one needs to at least count anyone denied existing known-to-be-safe-and-effective life-saving medicine, and those for whom denial is the result of expected delays and compensation schemes preventing timely mass production. The question is the extent to which one counts those for whom treatments were never developed, or were only developed after they already died, because of the expectation of FDA inaction.
Thus, my answers are:
For an unreasonable narrow interpretation that only counts those for whom the medicine was already sitting in a warehouse waiting for approval, and treat that shortage as a ‘whoops, making things is hard and takes time’ rather than a directly caused effect, the FDA is going to directly murder about 5,000 people in the United States.
For a compromise position where we count those who died while the FDA was holding up approval that would have been saved, the FDA is going to murder about 30,000 people in the United States if you start now, somewhat more if you start when we learned Paxlovid was safe and effective minus reasonable time to confirm this.
For a position that is comparing to the counterfactual of what would have happened if Pfizer knew in March 2020 that any pill it developed would be given out whenever there was sufficient Bayesian evidence it was safe and effective, and that they would be well-compensated for having more pills available quickly, the death toll is much higher, and especially much higher overseas.
As an exercise, one might want to consider the same calculations to delays in approval of the vaccines, remembering that Moderna and Pfizer both had their vaccine candidates within two days of starting work. And one could also do the same calculation on the continuing failure to approve and allow tests. Or one could include all the psychological and economic damage that could have been prevented, or the lives not fully lived, while we are forced to wait.
This is not Scott’s language, nor is it to ascribe murderous madness to any particular person. It is still important to call things by their right names.
One may wish to consider that perhaps this system of anti-ethics is part of a broader system that is also anti-epistemic, anti-virtue and anti-value in general, and that there is what might be called The Implicit Conspiracy that implicitly cooperates with and rewards with power those who demonstrate their dedication to this reversed system, and punishes those who fail to demonstrate this dedication or reward those who do show it. Would the resulting world look like our world?
Paxlovid Remains Illegal: 11/24 Update
Link post
What Alex Tabarrok called The Paxlovid Paradox is getting noticed by the people who notice such things, and completely ignored by everyone else. I’ve split off this week’s Paxlovid update to help make the situation easier to notice, and easier to remember and reference later, lest we forget.
Supply and Demand
The good news is that we are confident Paxlovid is safe and effective and the purchasing department is acting accordingly. Once we decide it is legal, we’ve secured our supply. The Biden administration has agreed to pay $5 billion for 10 million treatments. I previously thought this was 10 million pills as did the source, but I’ve been informed it was 10 million treatments, which means I was about to be happy to pay ten times what we’re actually paying. Kind of neat.
They’re going to make a profit from saving people’s lives. Outrageous!
The bad news, of course, is that Paxlovid Remains Illegal.
The Paxlovid Paradox
Scott Alexander is the latest to point out that this is murderous madness1. As I observed, and was later quoted at Marginal Revolution:
Scott Alexander notes that the current prediction market median approval date is January 1, 2022, with a 92% chance of approval by March. I’ve asked Polymarket to put up a market I would trust somewhat more, which should be up in a few days. My guess is that this the current median estimate is somewhat too pessimistic.
He goes through and quickly destroys many plausible reasons why this delay would make sense. In the comments several attempts were made to raise additional objections, with concerns about the ‘manufacturing process’ being the closest to kind of maybe being able to pretend to be a real consideration (as opposed to the others, which were at best pretending to pretend). I still don’t think it does an especially good job of pretending to be a big enough concern to matter, and was disappointed by Scott’s respect for the objection, which I’ll discuss more in detail below. Whatever pills we have now were somehow manufactured, without expected delays we would have made a lot more of them a lot faster, and we could have dealt with that issue while doing the trial if it was the true bottleneck and the incentives had been reasonable.
This is noteworthy, not because I think it’s identifying the causal mechanism (I think it mostly isn’t) but because it describes a world that makes more sense and stopping the trial unethical but a reasonable response to incentives. If only this were about people who responded so well to pure dollar incentives our problems would be so much easier to fix.
The Body Count
How bad is this delay? How many people are going to needlessly die?
Will is right. This is not hyperbole. It sure looks like a lot of people are going to needlessly die.
This is a system of anti-ethics, the reverse of actual ethics, because supporting rules in opposition to ethics shows your loyalties won’t be threatened by concerns about actual ethics.2
It is not hyperbole, but it is imprecise. What’s our best guess of how many Americans will die as a result of the FDA’s inaction on Paxlovid?
For a given person, Paxlovid saves their life if and only if:
They get Covid-19 and would have died without Paxlovid.
Covid-19 gets detected early enough to use Paxlovid.
They are given Paxlovid.
They live.
We would need to subtract any opportunity costs or side effects from Paxlovid, but for now it seems reasonable to assume the side effects are small when used on Covid-19 positive patients when compared to the benefits, and that the opportunity costs are also small since Paxlovid should combine well with other known-to-be-effective options.
We need to estimate the number of people who are going to die between now and approval in order to do step one. My guess is that 50,000 is somewhat high, since deaths lag and the surge is only starting now, so if we are starting now I’d use more like 40,000. But it’s better to start at least two weeks ago if we’re computing the overall death toll, so it’s more like 60,000 for those purposes. For now, we’re going to assume that ‘as soon as the clinical trial data is clear as day plus enough time to call and have a meeting’ is a reasonable starting point.
Step four only works about 90% of the time, so we need to multiply the number of dead people by 90%, as the other 10% couldn’t have been saved. That’s the other easy one.
Then we need to figure out how many of those who die detect Covid-19 fast enough to get Paxlovid when it would be effective. That’s trickier, especially with our current lack of tests, although that’s also the FDA’s fault. The failure rate that matters is that among cases that would be fatal without Paxlovid, so there’s biases pointing both ways—cases that are are harder to detect are mostly harmless, but cases where the patient doesn’t seek treatment fast enough tend to be deadlier. Given that our best guess is that most cases are never detected, the first consideration dominates. Also the existence of Paxlovid would make people more inclined to get tested. What is our best guess as to the Paxlovid-world’s detection rate? I’m going to guess this is something like 90% as well, the window is reasonably wide, but I’d also believe substantially lower, so I’ll use 80% to cover model uncertainty.
That leaves the big question. If the FDA approved Paxlovid, how many doses would be available? Would they be distributed to the people who needed them? Would those doses represent doses that would not otherwise be used, or would they move consumption forward that would have happened anyway?
This is trickier, and I don’t have the answer if we look only using causal decision theory on this particular choice. Paxlovid is 90% effective but the effectiveness per pill is many orders of magnitude lower since most people would recover anyway, and there were only 7 deaths in the entire clinical trial control group despite them enrolling over 1,000 patients.
It’s entirely possible that, while making Paxlovid widely available today instead of widely available on January 1 would save about 30,000 lives, given the delays already built into the system, the delay on approval won’t on net cost many lives. If it’s true that Pfizer will have about 180,000 treatments by end of year (Tweet was corrected), and each patient takes ten pills, then we could only treat 180,000 patients, and that probably saves about 5,000 lives (depends on how selective you think we can be in finding a vulnerable group, we can do better than the study but earlier my estimate of this was clearly too high and I was thinking 20k), less if the bottleneck would have continued for a while, but given how close it is the bottleneck presumably wouldn’t last too much longer. Which is only a moderate decrease.
This is relevant to the value of doing an end run of sorts around this particular decision point, or leaning on the FDA regarding it, if you don’t count impacts on future other decisions.
Responding to Incentives
That still treats the manufacturing process as if it couldn’t have gone any faster, or at least that the decisions made in how fast to go weren’t interlinked with expected future FDA behavior, at least in the past. If Pfizer would have expected to FDA to approve Paxlovid a month sooner, that’s a strong reason to get more pills made faster, even when there isn’t yet full confidence that the drug is going to work. Similarly, an advance commitment to paying a higher price for the first however many pills delivered quickly, conditional on the drug working, and ideally to also cover some costs if it failed, would have done a similar thing.
It’s a magician’s trick. The FDA causes the delays, and not in a strange functional decision theory or acausal trade kind of way. They do it in the direct obvious kind of way. It’s worth quoting the rest of Eliezer’s thread.
If one is counting the invisible graveyard now that it is about to both grow far larger and be much more highly visible, one needs to at least count anyone denied existing known-to-be-safe-and-effective life-saving medicine, and those for whom denial is the result of expected delays and compensation schemes preventing timely mass production. The question is the extent to which one counts those for whom treatments were never developed, or were only developed after they already died, because of the expectation of FDA inaction.
Thus, my answers are:
For an unreasonable narrow interpretation that only counts those for whom the medicine was already sitting in a warehouse waiting for approval, and treat that shortage as a ‘whoops, making things is hard and takes time’ rather than a directly caused effect, the FDA is going to directly murder about 5,000 people in the United States.
For a compromise position where we count those who died while the FDA was holding up approval that would have been saved, the FDA is going to murder about 30,000 people in the United States if you start now, somewhat more if you start when we learned Paxlovid was safe and effective minus reasonable time to confirm this.
For a position that is comparing to the counterfactual of what would have happened if Pfizer knew in March 2020 that any pill it developed would be given out whenever there was sufficient Bayesian evidence it was safe and effective, and that they would be well-compensated for having more pills available quickly, the death toll is much higher, and especially much higher overseas.
As an exercise, one might want to consider the same calculations to delays in approval of the vaccines, remembering that Moderna and Pfizer both had their vaccine candidates within two days of starting work. And one could also do the same calculation on the continuing failure to approve and allow tests. Or one could include all the psychological and economic damage that could have been prevented, or the lives not fully lived, while we are forced to wait.
Footnotes (not working right yet, sorry):
1
This is not Scott’s language, nor is it to ascribe murderous madness to any particular person. It is still important to call things by their right names.
2
One may wish to consider that perhaps this system of anti-ethics is part of a broader system that is also anti-epistemic, anti-virtue and anti-value in general, and that there is what might be called The Implicit Conspiracy that implicitly cooperates with and rewards with power those who demonstrate their dedication to this reversed system, and punishes those who fail to demonstrate this dedication or reward those who do show it. Would the resulting world look like our world?