As Covid is for the most part over and tests have not been in short supply for a very long time now, eliminating most of the rationale for complicated testing procedures meant to optimize test count/cost, it would be interesting for anyone who was involved in this to do a retrospective or post-mortem. Was pool testing useful? Why or why not? Can it be improved for future pandemics and should we care about it?
My impression, not having paid much attention to the details at the time (AI scaling was more important), is that various group/pool testing approaches were useful in a few niches, but were rendered useless in many places due to politics/organization/social flaws, and didn’t apply to most places.
I read that use of pool testing was common in China early on in 2020 as part of the hotspot strategy to economize on testing extremely large groups of people where only a few would have active Covid. This is the most logical place to apply it, and it was a major government & societal priority, so that is a success.
I get the impression that they were able to switch to individual testing fairly early, in order to get faster turnaround and do more localized testing. But to the extent that pool testing was effective from the start, before testing manufacturing could scale up, and helped squash Covid when it was still squashable, then it was very valuable even if obsoleted.
Of course, this evaluation can change depending on how Covid ultimately shakes out in China. Was it part of a heavyhanded but ultimately highly effective Covid Zero policy which led to it turning into a hermit country enduring devastating economic fallout while merely postponed the inevitable variant which could defeat Covid Zero? In which case, pool testing was a tool which played a part in buying China entire years to vaccinate but its good was rendered useless when they bizarrely failed to make any good use of the time, dabbling in ineffective vaccines etc, and in practice, only doubling down repeatedly on coercive testing and surveillance until the martingale blew up in their face. Or was it part of a visionary resolute effort which spared China the mass death of the rest of the world as it came out the other side by persisting so long in country-wide lockdown until Covid just sorta stopped being a problem somehow? Then its good was unalloyed and it’s sad so few other countries could boast a better record (eg Taiwan; I dunno if they made any use of it).
Sewage/water testing in places like colleges: various universities or towns made heavy use of what is in effect pool testing to do mass screening of water to detect infections and then hotspot them. This was only useful as long as you have a baseline of ~0, otherwise testing water merely gives you a proxy for overall infections, which is a lot less useful than using it to detect & squash hotspots. In individual cases of orgs with great discipline, this apparently worked well, and avoided the need for constant individual testing (which was expensive, not possible early on, and still inadequate—as all the infections inside ‘bubbles’ show, just too much leakage, test error, incubation, idiosyncrasies and so on).
Since most of these places were in countries where Covid became endemic eventually anyway, and everyone got infected, and in cases like colleges the young students were the last to get vaccinated (because they are the least harmed) and so needed to buy a lot of time to avoid infection before vaccination (which probably didn’t happen for most of them), these success stories seem to boil down to just buying time, which is a good deal less impressive.
At the margin, doesn’t seem like improvements to pool testing would have helped much compared to other things like faster vaccination approval.
Was it part of a heavyhanded but ultimately highly effective Covid Zero policy which led to it turning into a hermit country enduring devastating economic fallout while merely postponed the inevitable variant which could defeat Covid Zero?
Strong vote in favor of this interpretation of China’s policies. Doesn’t give me much to say about the post-mortem, though.
I’m catching up on Zvi’s COVID link roundups and it looks like, very unfortunately, the pessimistic interpretation has turned out to be right. What was Xi’s big COVID Zero exit strategy, what was his end game—how did China get from here to there when they refused to buy the best vaccines or ensure universal vaccination or stockpile Paxlovid or stagger infections by variolation or bulk up the healthcare system or… [many things which are more useful than security theater like spraying the streets with bleach or killing people’s pets]? How do you stop riding the tiger?* Turns out, there was no exit strategy. And now it’s moot.
Pro-China advocates (Naomi Wu’s tweets kept getting recommended to me, for some reason, although I doubt she was particularly notable or extreme so shouldn’t be singled out) downplayed the doomer forecasts that Omicron was eventually going to eat through ‘COVID Zero’ & praised its success. Doomers forecast that the longer ‘COVID Zero’ was enforced, the closer China would come to a breaking point, that Hong Kong was the harbinger of what was to come, and the results might be disastrous. It hadn’t happened yet, and might not for a long time, but with no exit strategy, the results would be real bad Real Soon Now™.
Aaaaaand… that appears to be what has happened last month. They could have ended COVID Zero at a time and place of their choosing, after having prepared for it; they chose ‘end it never’, which was not an option, and so it ended at a time not of their choosing. As the joke about bankruptcy (or any other exponential process) goes, how does it happen? “First slowly, then suddenly.” COVID Zero failed slowly, then suddenly. There are some interesting details about the protests and suppression thereof and whatnot, but that’s the summary: China suppressed COVID until it didn’t, and then exponentials gonna exponential.
The CCP is falsifying statistics now so it’s going to be hard to ever get certainty on how many deaths are happening. (Age-pyramid approaches won’t work because deaths are concentrated in the elderly, and the use of excess all-cause mortality is sufficiently well known now that they may falsify that as well, so the old approaches to figuring out the real death toll from all the CCP coverups like the Cultural Revolution or Great Leap Forward won’t work.) Nevertheless, the reporting on the ground plus the reasonably well understood COVID death rate mechanics implies things are bad. And there is no going back to COVID Zero after they have “let’er rip” like this. (And then there’s the reinfections! Fun.) It’s not clear that China is going to wind up with all that noticeably better a report card than many countries (although I’m sure they are already working hard to argue that it was better than the USA’s).
It’s a fascinating, if horrific, example of the incompetence and perversities of a totalitarian dictatorship: despite the most intense panopticon in human history, they still suffer from the same internal information-making and public choice pathologies as the others, and decision-making is brittle and in the long run, incompetent. “What if Xi Jinping [or Putin...] just isn’t that competent?” (This also connects to the CCP response to the chip embargo, which was first to handwave it away, then to announce a huge bailout, and then to… never mind, just cancel it and concede defeat?) Fukuyama undefeated—liberal democracy remains the worst system in the world, and the USA the worst country in the world, except for all the others that come up from time to time.
So, group testing failed in the sense that most of its value was contingent on enabling China to ride the tiger and figure out a way to get off it safely; but it didn’t. Ex ante, a good idea; ex post, a much less good idea.
My summary: group testing is a good technical quickfix but one whose value was completely bottlenecked through squishier political/organizational limiting steps, and in the case of COVID-19, those limiting steps limited it to ~0 net impact, and it represents a cautionary lesson for technocrats.
* application to other x-risks like nuclear war or AI risk left as an exercise to the reader: yes, ‘slow down AI capabilities progress’ to buy time, great, yes, sure, I agree—but buy time to do what?
for a long time, it was obvious that Zero Covid was not working, especially once increasingly infectious variants meant that there was community spread & all the measures had failed to make r<<1 but was hovering at r=1, and they were burning through patience, time, and money while denying all possibility of failure while also having no exit strategy; but it was equally obvious that people saying that were wrong, and Zero Covid was working, because after all, very few people were getting sick or dying of Covid. Every day, the pro-CCP people would say, ‘who are you going to believe, your lying eyes or prophets whose forecasts have never yet come true?’ and they would have a good point, as catastrophe failed to manifest for another day. And this would go on for days upon months upon years: anyone who said Zero Covid had to collapse relatively soon (keeping in mind ‘there is a great deal of ruin in a nation’ and things like bankruptcies or titanic frauds like Wirecard can always go on a lot longer than you think) would experience being wrong anywhere up to a good 700 times in a row, also with zero visible ‘progress’ to show. That is, ‘progress’ looked like things like COVID cases going above zero, and climbing log graphs, and community spread: but this is not what failure ‘looks like’ to regular people—to them, that just looks like more Zero Covid success.
But then December 2022 happened. And boom, almost literally overnight, Zero Covid was dropped and they let’er rip in just about the most destructive and harmful fashion possible one could drop Zero Covid. And then the pro-CCP forecasts lost all their forecasting points and the anti-ZC forecasts won bigly. But they were still wrong in important ways—as cynical as I was and sure that Zero Covid was going to fail at some point, I definitely didn’t take seriously ‘what if they just decide to stop Zero Covid overnight in the worst way possible, and there is no prolonged multi-month stepdown or long interval where they do all the sane sensible preparations that they have had such ample leisure to engage in?’ I wouldn’t’ve set up a prediction market on that, or if I had, I would have framed it in a modest sensible way which would’ve rendered it pointless.
Obviously, ‘foom’ or ‘hard takeoff’ is in a similar position to Hanson or Christiano-style positions. It will be ‘wrong’ (and you see a lot of this going around, especially on Twitter where there are many vested interests in deprecating AI risk), until it is right. The signs that foom is correct are easy to write off. For example, Hanson’s views on deep learning have been badly wrong many times: I tried to explain to him about transfer learning starting to work back in 2015 or so (a phenomenon I regarded as extremely important and which has in fact become so dominant in DL we take it utterly for granted) and he denied it with the usual Hansonian rebuttals; or when he denied that DL could scale at all, he mostly just ignored the early scaling work I linked him to like Hestness et al 2017. Or consider Transformers: a lynchpin of his position was that algorithms have to be heavily tailored to every domain and problem they are applied to, like they were in ML at that time—an AlphaGo in DRL had nothing to do with a tree search chess engine, much less stuff like Markov random fields in NLP, and this was just a fact of nature, and Yudkowsky’s fanciful vaporing about ‘general algorithms’ or ‘general intelligence’ so much wishful thinking. Then Transformers+scaling hit and here we are… Or how about his fight to the bitter end to pretend that ‘ems’ are ever going to be a thing and might beat DL to making AGI, despite near-zero progress (and the only visible progress both due to and would eventually benefit DL much more)? He was right for a long time, until he was wrong. None of this has provoked any substantial rethinking of his views, that I can tell: his latest post just doubles down on everything as if it was still 2013. Or the much more odious cases like Marcus, where they produce an example of a problem that DL will never be able to do—explicitly saying that, not merely insinuating it—and then when DL does it in a year or two, simply never referring to it again and coming up with a new problem DL will never be able to do; one would ask, why the new problem is any more relevant than the old problem was, and why did you ever put forward the old problem if you are completely untroubled by its solution, but that would ascribe more intellectual honesty to their exercise in FUD than it ever possessed.
But on the other hand, I don’t know what sensible forecasts I could’ve made about scaling that would be meaningful evidence right now and convince people, as opposed to being subtly wrong or too vague or morally right but marked wrong. When I got really excited reading Hestness, or Hinton on dark knowledge in JFT-300M, should I have registered a forecast: “in 2022, I prophesy that we will be training DL models with billions of parameters on billions of images!”? That would have been realistic just by drawing the line on the famous ‘AI and compute’ graph, it would turn out to have been very true, and yet—it’s not ‘cruxy’. No one cares if I did or did not make that prediction. Everyone who thought that was dumb back in 2017 because “DL doesn’t ‘just scale™’, we need many paradigm breakthroughs taking decades to reach these levels of performance” has gradually updated and indeed forgot their skepticism entirely or merely laugh about it gently, and has devised their new fallback positions about why AGI is safely far away—pointing out to them the correct prediction won’t change their minds about anything. And this seems to be true of everything else that I could have predicted in 2017.
We can’t solve this with the standard forecasting and prediction markets. They are too thinly traded, the base probabilities too low and too far into systematic bias territory, the contracts too long term, too likely to be misworded, and many key events could easily not happen and are too reliant on ‘catching a falling knife’ and issues with ‘the market can remain irrational longer than you can remain insolvent’/‘there is a great deal of ruin in a nation’. (Marcus would still be wrong even if no one had ever gotten around to retrying his GPT-2 examples on GPT-3, and Zero Covid would still be doomed even if it had managed to last a few % longer and was only going to collapse tomorrow rather than 3 months ago.)
This is why I spend more time thinking about capabilities and how they work than about the many repeated demands for dates or participation in PMs. We have a decent handle on how to grade answers to questions as good or bad; but what we really need right now are not good answers, but good questions.
“I tried to explain to him about transfer learning starting to work back in 2015 or so (a phenomenon I regarded as extremely important and which has in fact become so dominant in DL we take it utterly for granted) and he denied it with the usual Hansonian rebuttals; or when he denied that DL could scale at all, he mostly just ignored the early scaling work I linked him to like Hestness et al 2017. Or consider Transformers: a lynchpin of his position was that algorithms have to be heavily tailored to every domain and problem they are applied to, like they were in ML at that time—an AlphaGo in DRL had nothing to do with a tree search chess engine, much less stuff like Markov random fields in NLP, and this was just a fact of nature, and Yudkowsky’s fanciful vaporing about ‘general algorithms’ or ‘general intelligence’ so much wishful thinking. Then Transformers+scaling hit and here we are…”
I can’t find where you’ve had this exchange with him—can you find it?
If his embarrassing mistakes (and refusal to own up to them) is documented and demonstrable, why not just blam them onto his blog and twitter?
these success stories seem to boil down to just buying time, which is a good deal less impressive.
The counterpart to ‘faster vaccination approval’ is ‘buying time’ though. (Whether or not it ends up being well used, it is good at the time. The other reason to focus on it is—how much can you affect pool testing versus vaccination approval speed? Other stuff like improving statistical techniques might be easier for a lot of people than changing a specific organization.
[Question] COVID-19 Group Testing Post-mortem?
As Covid is for the most part over and tests have not been in short supply for a very long time now, eliminating most of the rationale for complicated testing procedures meant to optimize test count/cost, it would be interesting for anyone who was involved in this to do a retrospective or post-mortem. Was pool testing useful? Why or why not? Can it be improved for future pandemics and should we care about it?
My impression, not having paid much attention to the details at the time (AI scaling was more important), is that various group/pool testing approaches were useful in a few niches, but were rendered useless in many places due to politics/organization/social flaws, and didn’t apply to most places.
I read that use of pool testing was common in China early on in 2020 as part of the hotspot strategy to economize on testing extremely large groups of people where only a few would have active Covid. This is the most logical place to apply it, and it was a major government & societal priority, so that is a success.
I get the impression that they were able to switch to individual testing fairly early, in order to get faster turnaround and do more localized testing. But to the extent that pool testing was effective from the start, before testing manufacturing could scale up, and helped squash Covid when it was still squashable, then it was very valuable even if obsoleted.
Of course, this evaluation can change depending on how Covid ultimately shakes out in China. Was it part of a heavyhanded but ultimately highly effective Covid Zero policy which led to it turning into a hermit country enduring devastating economic fallout while merely postponed the inevitable variant which could defeat Covid Zero? In which case, pool testing was a tool which played a part in buying China entire years to vaccinate but its good was rendered useless when they bizarrely failed to make any good use of the time, dabbling in ineffective vaccines etc, and in practice, only doubling down repeatedly on coercive testing and surveillance until the martingale blew up in their face. Or was it part of a visionary resolute effort which spared China the mass death of the rest of the world as it came out the other side by persisting so long in country-wide lockdown until Covid just sorta stopped being a problem somehow? Then its good was unalloyed and it’s sad so few other countries could boast a better record (eg Taiwan; I dunno if they made any use of it).
Sewage/water testing in places like colleges: various universities or towns made heavy use of what is in effect pool testing to do mass screening of water to detect infections and then hotspot them. This was only useful as long as you have a baseline of ~0, otherwise testing water merely gives you a proxy for overall infections, which is a lot less useful than using it to detect & squash hotspots. In individual cases of orgs with great discipline, this apparently worked well, and avoided the need for constant individual testing (which was expensive, not possible early on, and still inadequate—as all the infections inside ‘bubbles’ show, just too much leakage, test error, incubation, idiosyncrasies and so on).
Since most of these places were in countries where Covid became endemic eventually anyway, and everyone got infected, and in cases like colleges the young students were the last to get vaccinated (because they are the least harmed) and so needed to buy a lot of time to avoid infection before vaccination (which probably didn’t happen for most of them), these success stories seem to boil down to just buying time, which is a good deal less impressive.
At the margin, doesn’t seem like improvements to pool testing would have helped much compared to other things like faster vaccination approval.
Strong vote in favor of this interpretation of China’s policies. Doesn’t give me much to say about the post-mortem, though.
I’m catching up on Zvi’s COVID link roundups and it looks like, very unfortunately, the pessimistic interpretation has turned out to be right. What was Xi’s big COVID Zero exit strategy, what was his end game—how did China get from here to there when they refused to buy the best vaccines or ensure universal vaccination or stockpile Paxlovid or stagger infections by variolation or bulk up the healthcare system or… [many things which are more useful than security theater like spraying the streets with bleach or killing people’s pets]? How do you stop riding the tiger?* Turns out, there was no exit strategy. And now it’s moot.
Pro-China advocates (Naomi Wu’s tweets kept getting recommended to me, for some reason, although I doubt she was particularly notable or extreme so shouldn’t be singled out) downplayed the doomer forecasts that Omicron was eventually going to eat through ‘COVID Zero’ & praised its success. Doomers forecast that the longer ‘COVID Zero’ was enforced, the closer China would come to a breaking point, that Hong Kong was the harbinger of what was to come, and the results might be disastrous. It hadn’t happened yet, and might not for a long time, but with no exit strategy, the results would be real bad Real Soon Now™.
Aaaaaand… that appears to be what has happened last month. They could have ended COVID Zero at a time and place of their choosing, after having prepared for it; they chose ‘end it never’, which was not an option, and so it ended at a time not of their choosing. As the joke about bankruptcy (or any other exponential process) goes, how does it happen? “First slowly, then suddenly.” COVID Zero failed slowly, then suddenly. There are some interesting details about the protests and suppression thereof and whatnot, but that’s the summary: China suppressed COVID until it didn’t, and then exponentials gonna exponential.
The CCP is falsifying statistics now so it’s going to be hard to ever get certainty on how many deaths are happening. (Age-pyramid approaches won’t work because deaths are concentrated in the elderly, and the use of excess all-cause mortality is sufficiently well known now that they may falsify that as well, so the old approaches to figuring out the real death toll from all the CCP coverups like the Cultural Revolution or Great Leap Forward won’t work.) Nevertheless, the reporting on the ground plus the reasonably well understood COVID death rate mechanics implies things are bad. And there is no going back to COVID Zero after they have “let’er rip” like this. (And then there’s the reinfections! Fun.) It’s not clear that China is going to wind up with all that noticeably better a report card than many countries (although I’m sure they are already working hard to argue that it was better than the USA’s).
It’s a fascinating, if horrific, example of the incompetence and perversities of a totalitarian dictatorship: despite the most intense panopticon in human history, they still suffer from the same internal information-making and public choice pathologies as the others, and decision-making is brittle and in the long run, incompetent. “What if Xi Jinping [or Putin...] just isn’t that competent?” (This also connects to the CCP response to the chip embargo, which was first to handwave it away, then to announce a huge bailout, and then to… never mind, just cancel it and concede defeat?) Fukuyama undefeated—liberal democracy remains the worst system in the world, and the USA the worst country in the world, except for all the others that come up from time to time.
So, group testing failed in the sense that most of its value was contingent on enabling China to ride the tiger and figure out a way to get off it safely; but it didn’t. Ex ante, a good idea; ex post, a much less good idea.
My summary: group testing is a good technical quickfix but one whose value was completely bottlenecked through squishier political/organizational limiting steps, and in the case of COVID-19, those limiting steps limited it to ~0 net impact, and it represents a cautionary lesson for technocrats.
* application to other x-risks like nuclear war or AI risk left as an exercise to the reader: yes, ‘slow down AI capabilities progress’ to buy time, great, yes, sure, I agree—but buy time to do what?
Reading Dan Wang’s belated letter, where he describes Shanghai and the abrupt collapse of Zero Covid, reminds me of one interesting aspect for us of base rates, Outside View reasoning, rocks which say ’everything is fine, and the difference between probabilities & decisions:
for a long time, it was obvious that Zero Covid was not working, especially once increasingly infectious variants meant that there was community spread & all the measures had failed to make r<<1 but was hovering at r=1, and they were burning through patience, time, and money while denying all possibility of failure while also having no exit strategy; but it was equally obvious that people saying that were wrong, and Zero Covid was working, because after all, very few people were getting sick or dying of Covid. Every day, the pro-CCP people would say, ‘who are you going to believe, your lying eyes or prophets whose forecasts have never yet come true?’ and they would have a good point, as catastrophe failed to manifest for another day. And this would go on for days upon months upon years: anyone who said Zero Covid had to collapse relatively soon (keeping in mind ‘there is a great deal of ruin in a nation’ and things like bankruptcies or titanic frauds like Wirecard can always go on a lot longer than you think) would experience being wrong anywhere up to a good 700 times in a row, also with zero visible ‘progress’ to show. That is, ‘progress’ looked like things like COVID cases going above zero, and climbing log graphs, and community spread: but this is not what failure ‘looks like’ to regular people—to them, that just looks like more Zero Covid success.
But then December 2022 happened. And boom, almost literally overnight, Zero Covid was dropped and they let’er rip in just about the most destructive and harmful fashion possible one could drop Zero Covid. And then the pro-CCP forecasts lost all their forecasting points and the anti-ZC forecasts won bigly. But they were still wrong in important ways—as cynical as I was and sure that Zero Covid was going to fail at some point, I definitely didn’t take seriously ‘what if they just decide to stop Zero Covid overnight in the worst way possible, and there is no prolonged multi-month stepdown or long interval where they do all the sane sensible preparations that they have had such ample leisure to engage in?’ I wouldn’t’ve set up a prediction market on that, or if I had, I would have framed it in a modest sensible way which would’ve rendered it pointless.
Obviously, ‘foom’ or ‘hard takeoff’ is in a similar position to Hanson or Christiano-style positions. It will be ‘wrong’ (and you see a lot of this going around, especially on Twitter where there are many vested interests in deprecating AI risk), until it is right. The signs that foom is correct are easy to write off. For example, Hanson’s views on deep learning have been badly wrong many times: I tried to explain to him about transfer learning starting to work back in 2015 or so (a phenomenon I regarded as extremely important and which has in fact become so dominant in DL we take it utterly for granted) and he denied it with the usual Hansonian rebuttals; or when he denied that DL could scale at all, he mostly just ignored the early scaling work I linked him to like Hestness et al 2017. Or consider Transformers: a lynchpin of his position was that algorithms have to be heavily tailored to every domain and problem they are applied to, like they were in ML at that time—an AlphaGo in DRL had nothing to do with a tree search chess engine, much less stuff like Markov random fields in NLP, and this was just a fact of nature, and Yudkowsky’s fanciful vaporing about ‘general algorithms’ or ‘general intelligence’ so much wishful thinking. Then Transformers+scaling hit and here we are… Or how about his fight to the bitter end to pretend that ‘ems’ are ever going to be a thing and might beat DL to making AGI, despite near-zero progress (and the only visible progress both due to and would eventually benefit DL much more)? He was right for a long time, until he was wrong. None of this has provoked any substantial rethinking of his views, that I can tell: his latest post just doubles down on everything as if it was still 2013. Or the much more odious cases like Marcus, where they produce an example of a problem that DL will never be able to do—explicitly saying that, not merely insinuating it—and then when DL does it in a year or two, simply never referring to it again and coming up with a new problem DL will never be able to do; one would ask, why the new problem is any more relevant than the old problem was, and why did you ever put forward the old problem if you are completely untroubled by its solution, but that would ascribe more intellectual honesty to their exercise in FUD than it ever possessed.
But on the other hand, I don’t know what sensible forecasts I could’ve made about scaling that would be meaningful evidence right now and convince people, as opposed to being subtly wrong or too vague or morally right but marked wrong. When I got really excited reading Hestness, or Hinton on dark knowledge in JFT-300M, should I have registered a forecast: “in 2022, I prophesy that we will be training DL models with billions of parameters on billions of images!”? That would have been realistic just by drawing the line on the famous ‘AI and compute’ graph, it would turn out to have been very true, and yet—it’s not ‘cruxy’. No one cares if I did or did not make that prediction. Everyone who thought that was dumb back in 2017 because “DL doesn’t ‘just scale™’, we need many paradigm breakthroughs taking decades to reach these levels of performance” has gradually updated and indeed forgot their skepticism entirely or merely laugh about it gently, and has devised their new fallback positions about why AGI is safely far away—pointing out to them the correct prediction won’t change their minds about anything. And this seems to be true of everything else that I could have predicted in 2017.
We can’t solve this with the standard forecasting and prediction markets. They are too thinly traded, the base probabilities too low and too far into systematic bias territory, the contracts too long term, too likely to be misworded, and many key events could easily not happen and are too reliant on ‘catching a falling knife’ and issues with ‘the market can remain irrational longer than you can remain insolvent’/‘there is a great deal of ruin in a nation’. (Marcus would still be wrong even if no one had ever gotten around to retrying his GPT-2 examples on GPT-3, and Zero Covid would still be doomed even if it had managed to last a few % longer and was only going to collapse tomorrow rather than 3 months ago.)
This is why I spend more time thinking about capabilities and how they work than about the many repeated demands for dates or participation in PMs. We have a decent handle on how to grade answers to questions as good or bad; but what we really need right now are not good answers, but good questions.
“I tried to explain to him about transfer learning starting to work back in 2015 or so (a phenomenon I regarded as extremely important and which has in fact become so dominant in DL we take it utterly for granted) and he denied it with the usual Hansonian rebuttals; or when he denied that DL could scale at all, he mostly just ignored the early scaling work I linked him to like Hestness et al 2017. Or consider Transformers: a lynchpin of his position was that algorithms have to be heavily tailored to every domain and problem they are applied to, like they were in ML at that time—an AlphaGo in DRL had nothing to do with a tree search chess engine, much less stuff like Markov random fields in NLP, and this was just a fact of nature, and Yudkowsky’s fanciful vaporing about ‘general algorithms’ or ‘general intelligence’ so much wishful thinking. Then Transformers+scaling hit and here we are…”
I can’t find where you’ve had this exchange with him—can you find it?
If his embarrassing mistakes (and refusal to own up to them) is documented and demonstrable, why not just blam them onto his blog and twitter?
The counterpart to ‘faster vaccination approval’ is ‘buying time’ though. (Whether or not it ends up being well used, it is good at the time. The other reason to focus on it is—how much can you affect pool testing versus vaccination approval speed? Other stuff like improving statistical techniques might be easier for a lot of people than changing a specific organization.
How good would have been a pre-mortem as many products in tech get before they are released to the public? https://medium.com/paypal-tech/pre-mortem-technically-working-backwards-1724eafbba02