Sammy Bahia-Martin. Philosophy and Physics BSc, AI MSc at Edinburgh, starting a PhD at King’s College London. Interested in ethics, general philosophy and AI Safety.
SDM
Karma: 913
According to this article (https://www.deplatformdisease.com/blog/addressing-geert-vanden-bossches-claims) - the key claim of (2), that natural antibodies are superior to vaccine antibodies and permanently replaced by them, is just wrong (‘absolute unvarnished nonsense’ was the quote). One or the other is right, and we just need someone who actually knows immunology to tell us
Principally, antibodies against SARS-CoV-2 could be of value if they are neutralizing. Bossche presents no evidence to support that natural IgM is neutralizing (rather than just binding) SARS-CoV-2.
We have had quite significant news accumulating this week; Delta (Indian) COVID-19 has been shown to be (central estimate) 2.5 times deadlier than existing strains and the central estimate of its increased transmissibility is down a bit, but still 50-70% on top of B.1.1.7 (just imagine if we’d had to deal with Delta in early 2020!), though with a fairly small vaccine escape. This threadgives a good summary of the modelling and likely consequences for the UK, and also more or less applies to most countries with high vaccination rates like the US.
For the US/UK this situation is not too concerning, certainly not compared to March or December 2020, and if restrictions are held where they are now the R_t will likely soon go under 1 as vaccination rates increase. However, there absolutely can be a large exit wave that could push up hospitalizations and in the reasonable worst case lead to another lockdown. Also, the outlook for the rest of the world is significantly worse than it looked just a month ago thanks to this variant—see this by Zeynep Turfecki.
The data is preliminary, and I really hope that the final estimate ends up as low as possible. But coupled with what we are observing in India and in Nepal, where it is rampant, I fear that the variant is a genuine threat.
In practical terms, to put it bluntly, it means that the odds that the pandemic will end because enough people have immunity via getting infected rather than being vaccinated just went way up.
Effective Altruists may want to look to India-like oxygen interventions in other countries over the next couple of months.
You cover most of the interesting possibilities on the military technology front, but one thing that you don’t mention that might matter especially considering the recent near-breakdowns of some of the nuclear weapon treaties e.g. NEWSTART, is the further proliferation of nuclear weapons including fourth generation nuclear weapons like nuclear shaped charge warheads, pure fusion and sub-kiloton devices or tactical nuclear weapons—and more countries fitting nuclear-armed cruise missiles or drones with nuclear capability which might be a destabilising factor. If laser technology is sufficiently developed we may also see other forms of directed energy weapons becoming more common such as electron beam weapons or electrolasers
For anyone reading, please consider following in Vitalik’s footsteps and donating to the GiveIndia Oxygen fundraiser, which likely beats givewell’s top charities in terms of life-years saved per dollar.
One of the more positive signs that I’ve seen in recent times, is that well-informed elite opinion (going by, for example, the Economist editorials) has started to shift towards scepticism of institutions and a recognition of how badly they’ve failed. Among the people who matter for policymaking, the scale of the failure has not been swept under the rug. See here:
We believe that Mr Biden is wrong. A waiver may signal that his administration cares about the world, but it is at best an empty gesture and at worst a cynical one.
...
Economists’ central estimate for the direct value of a course is $2,900—if you include factors like long covid and the effect of impaired education, the total is much bigger.
This strikes me as the sort of remark I’d expect to see in one of these comment threads, which has to be a good sign.
In that same issue, we also saw the first serious attempt that I’ve seen to calculate the total death toll of Covid, accounting for all reporting biases, throughout the world. The Economist was the only publication I’ve seen that didn’t parrot the almost-meaningless official death toll figures. The true answer is, of course, horrifying: between 7.1m and 12.7m dead, with a central estimate of 10.2m—this unfortunately means that we ended up with the worst case scenario I imagined back in late February. Moreover, we appear to currently be at the deadliest point of the entire pandemic.
Great post! I’m glad someone has outlined in clear terms what these failures look like, rather than the nebulous ‘multiagent misalignment’, as it lets us start on a path to clarifying what (if any) new mitigations or technical research are needed.
Agent-agnostic perspective is a very good innovation for thinking about these problems—is line between agentive and non-agentive behaviour is often not clear, and it’s not like there is a principled metaphysical distinction between the two (e.g. Dennett and the Intentional Stance). Currently, big corporations can be weakly modelled this way and individual humans are fully agentive, but Transformative AI will bring up a whole spectrum of more and less agentive things that will fill up the rest of this spectrum.
There is a sense in which, if the outcome is something catastrophic, there must have been misalignment, and if there was misalignment then in some sense at least some individual agents were misaligned. Specifically, the systems in your Production Web weren’t intent-aligned because they weren’t doing what we wanted them to do, and were at least partly deceiving us. Assuming this is the case, ‘multipolar failure’ requires some subset of intent misalignment. But it’s a special subset because it involves different kinds of failures to the ones we normally talk about.
It seems like you’re identifying some dimensions of intent alignment as those most likely to be neglected because they’re the hardest to catch, or because there will be economic incentives to ensure AI isn’t aligned in that way, rather than saying that there some sense in which the transformative AI in the production web scenario is ‘fully aligned’ but still produces an existential catastrophe.
I think that the difference between your Production Web and Paul Christiano’s subtle creeping Outer Alignment failure scenario is just semantic—you say that the AIs involved are aligned in some relevant sense while Christiano says they are misaligned.
The further question then becomes, how clear is the distinction between multiagent alignment and ‘all of alignment except multiagent alignment’. This is the part where your claim of ‘Problems before solutions’ actually does become an issue—given that the systems going wrong in Production Web aren’t Intent-aligned (I think you’d agree with this), at a high level the overall problem is the same in single and multiagent scenarios.
So for it to be clear that there is a separate multiagent problem to be solved, we have to have some reason to expect that the solutions currently intended to solve single agent intent alignment aren’t adequate, and that extra research aimed at examining the behaviour of AI e.g. in game theoretic situations, or computational social choice research, is required to avert these particular examples of misalignment.
A related point—as with single agent misalignment, the Fast scenarios seem more certain to occur, given their preconditions, than the slow scenarios.
A certain amount of stupidity and lack of coordination persisting for a while is required in all the slow scenarios, like the systems involved in Production Web being allowed to proliferate and be used more and more even if an opportunity to coordinate and shut the systems down exists and there are reasons to do so. There isn’t an exact historical analogy for that type of stupidity so far, though a few things come close (e.g. covid response, leadup to WW2, cuban missile crisis).
As with single agent fast takeoff scenarios, in the fast stories there is a key ‘treacherous turn’ moment where the systems suddenly go wrong, which requires much less lack of coordination to be plausible than the slow Production Web scenarios.
Therefore, multipolar failure is less dangerous if takeoff is slower, but the difference in risk between slow vs fast takeoff for multipolar failure is unfortunately a lot smaller than the slow vs fast risk difference for single agent failure (where the danger is minimal if takeoff is slow enough). So multiagent failures seem like they would be the dominant risk factor if takeoff is sufficiently slow.
Yes, its very oversimplified—in this case ‘capability’ just refers to whatever enables RSI, and we assume that it’s a single dimension. Of course, it isn’t, but we assume that the capability can be modelled this way as a very rough approximation.
Physical limits are another thing the model doesn’t cover—you’re right to point out that on the intelligence explosion/full RSI scenarios the graph goes vertical only for a time until some limit is hit
Update to ‘Modelling Continuous Progress’
I made an attempt to model intelligence explosion dynamics in this post, by attempting to make the very oversimplified exponential-returns-to-exponentially-increasing-intelligence model used by Bostrom and Yudkowsky slightly less oversimplified.
This post tries to build on a simplified mathematical model of takeoff which was first put forward by Eliezer Yudkowsky and then refined by Bostrom in Superintelligence, modifying it to account for the different assumptions behind continuous, fast progress as opposed to discontinuous progress. As far as I can tell, few people have touched these sorts of simple models since the early 2010’s, and no-one has tried to formalize how newer notions of continuous takeoff fit into them. I find that it is surprisingly easy to accommodate continuous progress and that the results are intuitive and fit with what has already been said qualitatively about continuous progress.
The page includes python code for the model.
This post doesn’t capture all the views of takeoff—in particular it doesn’t capture the non-hyperbolic faster growth mode scenario, where marginal intelligence improvements are exponentially increasingly difficult, and therefore we get a (continuous or discontinuous switch to a) new exponential growth mode rather than runaway hyperbolic growth.
But I think that by modifying the f(I) function that determines how RSI capability varies with intelligence we can incorporate such views.
In the context of the exponential model given in the post that would correspond to an f(I) function where
which would result in a continuous (determined by size of d) switch to a single faster exponential growth mode
But I think the model still roughly captures the intuition behind scenarios that involve either a continuous or a discontinuous step to an intelligence explosion.
Given the model assumptions, we see how the different scenarios look in practice:
If we plot potential AI capability over time, we can see how no new growth mode (brown) vs a new growth mode (all the rest), the presence of an intelligence explosion (red and orange) vs not (green and purple), and the presence of a discontinuity (red and purple) vs not (orange and green) affect the takeoff trajectory.
Is a bridge falling down the moment you finish building it an extreme and somewhat strange failure mode? In the space of all possible bridge designs, surely not. Most bridge designs fall over. But in the real world, you could win money all day betting that bridges won’t collapse the moment they’re finished.
I’m not saying this is an exact analogy for AGI alignment—there are lots of specific technical reasons to expect that alignment is not like bridge building and that there are reasons why the approaches we’re likely to try will break on us suddenly in ways we can’t fix as we go—treacherous turns, inner misalignment or reactions to distributional shift. It’s just that there are different answers to the question of what’s the default outcome depending on if you’re asking what to expect abstractly or in the context of how things are in fact done.
Instrumental Convergence plus a specific potential failure mode (like e.g. we won’t pay sufficient attention to out of distribution robustness), is like saying ‘you know the vast majority of physically possible bridge designs fall over straight away and also there’s a giant crack in that load-bearing concrete pillar over there’ - if for some reason your colleague has a mental block around the idea that a bridge could in principle fall down then the first part is needed (hence why IC is important for presentations of AGI risk because lots of people have crazily wrong intuitions about the nature of AI or intelligence), but otherwise IC doesn’t do much to help the case for expecting catastrophic misalignment and isn’t enough to establish that failure is a default outcome.
It seems like your reason for saying that catastrophic misalignment can’t be considered an abnormal or extreme failure mode comes down to this pre-technical-detail Instrumental Convergence thesis—that IC by itself gives us a significant reason to worry, even if we all agree that IC is not the whole story.
this seems a bizarre way to describe something that we agree is the default result of optimizing for almost anything (eg paperclips).
= ‘because strongly optimizing for almost anything leads to catastrophe via IC, we can’t call catastrophic misalignment a bizarre outcome’?
Maybe it’s just a subtle difference in emphasis without a real difference in expectation/world model, but I think there is an important need to clarify the difference between ‘IC alone raises an issue that might not be obvious but doesn’t give us a strong reason to expect a catastrophe’ and ‘IC alone suggests a catastrophe even though it’s not the whole story’ - and the first of these is a more accurate way of viewing the role of IC in establishing the likelihood of catastrophic misalignment.
Ben Garfinkel argues for the first of these and against the second, in his objection to the ‘classic’ formulation of instrumental convergence/orthogonality—that these are just ‘measure based’ arguments which identify that a majority of possible AI designs with some agentive properties and large-scale goals will optimize in malign ways, rather than establishing that we’re actually likely to build such agents.
I made an attempt to model intelligence explosion dynamics in this post, by attempting to make the very oversimplified exponential-returns-to-exponentially-increasing-intelligence model used by Bostrom and Yudkowsky slightly less oversimplified.
This post tries to build on a simplified mathematical model of takeoff which was first put forward by Eliezer Yudkowsky and then refined by Bostrom in Superintelligence, modifying it to account for the different assumptions behind continuous, fast progress as opposed to discontinuous progress. As far as I can tell, few people have touched these sorts of simple models since the early 2010’s, and no-one has tried to formalize how newer notions of continuous takeoff fit into them. I find that it is surprisingly easy to accommodate continuous progress and that the results are intuitive and fit with what has already been said qualitatively about continuous progress.
The page includes python code for the model.
This post doesn’t capture all the views of takeoff—in particular it doesn’t capture the non-hyperbolic faster growth mode scenario, where marginal intelligence improvements are exponentially increasingly difficult and therefore we get a (continuous or discontinuous switch to a) new exponential growth mode rather than runaway hyperbolic growth.
But I think that by modifying the f(I) function that determines how RSI capability varies with intelligence we can incorporate such views.
(In the context of the exponential model given in the post that would correspond to an f(I) function where
which would result in a continuous (determined by size of d) switch to a single faster exponential growth mode)
But I think the model still roughly captures the intuition behind scenarios that involve either a continuous or a discontinuous step to an intelligence explosion.
Modern literature about immortality is written primarily by authors who expect to die, and their grapes are accordingly sour.
This is still just as true as when this essay was written, I think—even the Culture had its human citizens mostly choosing to die after a time… to the extent that I eventually decided: if you want something done properly, do it yourself.
But there are exceptions—the best example of published popular fiction that has immortality as a basic fact of life is the Commonwealth Saga by Peter F Hamilton and the later Void Trilogy (the first couple of books were out in 2007).
The Commonwealth has effective immortality, a few downsides of it are even noticable (their culture and politics is a bit more stagnant than we might like), but there’s never any doubt at all that it’s worth it, and it’s barely commented on in the story,
In truth, I suspect that if people were immortal, they would not think overmuch about the meaning that immortality gives to life.
(Incidentally, the latter-day Void Trilogy Commonwealth is probably the closest a work of published fiction has come to depicting a true eudaimonic utopia that lacks the problems of the culture)
I wonder if there’s been any harder to detect shift in how immortality is portrayed in fiction since 2007? Is it still as rare now as then to depict it as a bad thing?
The UK vaccine rollout is considered a success, and by the standards of other results, it is indeed a success. This interview explains how they did it, which was essentially ‘make deals with companies and pay them money in exchange for doses of vaccines.’
A piece of this story you may find interesting (as an example of a government minister making a decision based on object level physical considerations): multiple reports say Matt Hancock, the UK’s health Secretary, made the decision to insist on over-ordering vaccines because he saw the movie Contagion and was shocked into viscerally realising how important a speedy rollout was.
It might just be a nice piece of PR, but even if that’s the case it’s still a good metaphor for how object level physical considerations can intrude into government decision making
I agree with your argument about likelihood of DSA being higher compared to previous accelerations, due to society not being able to speed up as fast as the technology. This is sorta what I had in mind with my original argument for DSA; I was thinking that leaks/spying/etc. would not speed up nearly as fast as the relevant AI tech speeds up.
Your post on ‘against GDP as a metric’ argues more forcefully for the same thing that I was arguing for, that
‘the economic doubling time’ stops being so meaningful—technological progress speeds up abruptly but other kinds of progress that adapt to tech progress have more of a lag before the increased technological progress also affects them?
So we’re on the same page there that it’s not likely that ‘the economic doubling time’ captures everything that’s going on all that well, which leads to another problem—how do we predict what level of capability is necessary for a transformative AI to obtain a DSA (or reach the PONR for a DSA)?
I notice that in your post you don’t propose an alternative metric to GDP, which is fair enough since most of your arguments seem to lead to the conclusion that it’s almost impossibly difficult to predict in advance what level of advantage over the rest of the world in which areas are actually needed to conquer the world, since we seem to be able to analogize persuasion tools to or conquistador-analogues who had relatively small tech advantages, to the AGI situation.
I think that there is still a useful role for raw economic power measurements, in that they provide a sort of upper bound on how much capability difference is needed to conquer the world. If an AGI acquires resources equivalent to controlling >50% of the world’s entire GDP, it can probably take over the world if it goes for the maximally brute force approach of just using direct military force. Presumably the PONR for that situation would be awhile before then, but at least we know that an advantage of a certain size would be big enough given no assumptions about the effectiveness of unproven technologies of persuasion or manipulation or specific vulnerabilities in human civilization.
So we can use our estimate of how doubling time may increase, anchor on that gap and estimate down based on how soon we think the PONR is, or how many ‘cheat’ pathways that don’t involve economic growth there are.
The whole idea of using brute economic advantage as an upper limit ‘anchor’ I got from Ajeya’s Post about using biological anchors to forecast what’s required for TAI—if we could find a reasonable lower bound for the amount of advantage needed to attain DSA we could do the same kind of estimated distribution between them. We would just need a lower limit—maybe there’s a way of estimating it based on the upper limit of human ability since we know no actually existing human has used persuasion to take over the world but as you point out they’ve come relatively close.
I realize that’s not a great method, but is there any better alternative given that this is a situation we’ve never encountered before, for trying to predict what level of capability is necessary for DSA? Or perhaps you just think that anchoring your prior estimate based on economic power advantage as an upper bound is so misleading it’s worse than having a completely ignorant prior. In that case, we might have to say that there are just so many unprecedented ways that a transformative AI could obtain a DSA that we can just have no idea in advance what capability is needed, which doesn’t feel quite right to me.
Finally got round to reading your sequence and it looks like we disagree a lot less than I thought, since your first three causes are exactly what I was arguing for in my reply,
This is probably the crux. I don’t think we tend to go to higher simulacra levels now, compared to decades ago. I think it’s always been quite prevalent, and has been roughly constant through history. While signalling explanations definitely tell us a lot about particular failings, they can’t explain the reason things are worse now in certain ways, compared to before. The difference isn’t because of the perennial problem of pervasive signalling. It has more to do with economic stagnation and not enough state capacity. These flaws mean useful action gets replaced by useless action, and allow more room for wasteful signalling.
As one point in favour of this model, I think it’s worth noting that the historical comparisons aren’t ever to us actually succeeding at dealing with pandemics in the past, but to things like “WWII-style” efforts—i.e. thinking that if we could just do x as well as we once did y then things would have been a lot better.
This implies that if you made an institution analogous to e.g. the weapons researchers of WW2 and the governments that funded them, or NASA in the 1960s, without copy-pasting 1940s/1960s society wholesale, the outcome would have been better. To me that suggests it’s institution design that’s the culprit, not this more ethereal value drift or increase in overall simulacra levels.
I think you’d agree with most of that, except that you see a much more significant causal role for the cultural factors like increased fragility and social atomisation. There is pretty solid evidence for both being real problems, Jon Haidt presents the best case to take these seriously, although it’s not as definitive as you make out (E.g. Suicide rates are basically a random walk), and your explanation for how they lead to institutional problems is reasonable, but I wonder if they are even needed as explanations when your first three causes are so strong and obvious,
Essentially I see your big list like this:
Main Drivers:
Cause 1: More Real Need For Large Organizations (includes decreasing low hanging fruit) Cause 2: Laws and Regulations Favor Large Organizations Cause 3: Less Disruption of Existing Organizations Cause 5: Rent Seeking is More Widespread and Seen as Legitimate
Real but more minor:
Cause 4: Increased Demand for Illusion of Safety and Security Cause 8: Atomization and the Delegitimization of Human Social Needs Cause 7: Ignorance Cause 9: Educational System Cause 10: Vicious Cycle
No idea but should look into:
Cause 6: Big Data, Machine Learning and Internet Economics
Essentially my view is that if you directly addressed the main drivers with large legal or institutional changes the other causes of mazedom wouldn’t fight back.
I believe that the ‘obvious legible institutional risks first’ view is in line with what others who’ve written on this problem like Tyler Cowen or Sam Bowman think, but it’s a fairly minor disagreement since most of your proposed fixes are on the institutional side of things anyway.
Also, the preface is very important—these are some of the only trends that seem to be going the wrong way consistently in developed countries for a while now, and they’re exactly the forces you’d expect to be hardest to resist.
The world is better for people than it was back then. There are many things that have improved. This is not one of them.
Currently the most plausible doom scenario in my mind is maybe a version of Paul’s Type II failure. (If this is surprising to you, reread it while asking yourself what terms like “correlated automation failure” are euphemisms for.)
This is interesting, and I’d like to see you expand on this. Incidentally I agree with the statement, but I can imagine both more and less explosive, catastrophic versions of ‘correlated automation failure’. On the one hand it makes me think of things like transportation and electricity going haywire, on the other it could fit a scenario where a collection of powerful AI systems simultaneously intentionally wipe out humanity.
Clock-time leads shrink automatically as the pace of innovation speeds up, because if everyone is innovating 10x faster, then you need 10x as many hoarded ideas to have an N-year lead.
What if, as a general fact, some kinds of progress (the technological kinds more closely correlated with AI) are just much more susceptible to speed-up? I.e, what if ‘the economic doubling time’ stops being so meaningful—technological progress speeds up abruptly but other kinds of progress that adapt to tech progress have more of a lag before the increased technological progress also affects them? In that case, if the parts of overall progress that affect the likelihood of leaks, theft and spying aren’t sped up by as much as the rate of actual technology progress, the likelihood of DSA could rise to be quite high compared to previous accelerations where the order of magnitude where the speed-up occurred was fast enough to allow society to ‘speed up’ the same way.
In other words—it becomes easier to hoard more and more ideas if the ability to hoard ideas is roughly constant but the pace of progress increases. Since a lot of these ‘technologies’ for facilitating leaks and spying are more in the social realm, this seems plausible.
But if you need to generate more ideas, this might just mean that if you have a very large initial lead, you can turn it into a DSA, which you still seem to agree with:
Even if takeoff takes several years it could be unevenly distributed such that (for example) 30% of the strategically relevant research progress happens in a single corporation. I think 30% of the strategically relevant research happening in a single corporation at beginning of a multi-year takeoff would probably be enough for DSA.
I meant, ‘based on what you’ve said about Zvi’s model’ I.e. Nostalgebraist says zvi says Rt never goes below 1 - if you look at the plot he produced Rt is always above 1 given Zvi’s assumptions, which the London data falsified.
It seems better to first propose a model we know can match past data, and then add a tuning term/effect for “pandemic fatigue” for future prediction.
To get a sense of scale, here is one of the plots from my notebook:
The colored points show historical data on R vs. the 6-period average, with color indicating the date.
Thanks for actually plotting historical Rt vs infection rates!
Whereas, it seems more natural to take (3) as evidence that (1) was wrong.
In my own comment, I also identified the control system model of any kind of proportionality of Rt to infections as a problem. Based on my own observations of behaviour and government response, the MNM hypothesis seems more likely (governments hitting the panic button as imminent death approaches, i.e. hospitals begin to be overwhelmed) than a response that ramps up proportionate to recent infections. I think that explains the tight oscillations.
I’d say the dominant contributor to control systems is something like a step function at a particular level near where hospitals are overwhelmed, and individual responses proportionate to exact levels of infection are a lesser part of it.
You could maybe operationalize this by looking at past hospitalization rates, fitting a logistic curve to them at the ‘overwhelmed’ threshold and seeing if that predicts Rt. I think it would do pretty well.
This tight control was a surprise and is hard to reproduce in a model, but if our model doesn’t reproduce it, we will go on being surprised by the same thing that surprised us before.
My own predictions are essentially based on continuing to expect the ‘tight control’ to continue somehow, i.e. flattening out cases or declining a bit at a very high level after a large swing upwards.
It looks like (subsequent couple of days data seem to confirm this), Rt is currently just below 1 in London—which would outright falsify any model that claims Rt never goes below 1 for any amount of infection with the new variant, given our control system response, which according to your graph, the infections exponential model does predict.
If you ran this model on the past, what would it predict? Based on what you’ve said, Rt never goes below one, so there would be a huge first wave with a rapid rise up to partial herd immunity over weeks, based on your diagram. That’s the exact same predictive error that was made last year.
I note—outside view—that this is very similar to the predictive mistake made last Febuary/March with old Covid-19 - many around here were practically certain we were bound for an immediate (in a month or two) enormous herd immunity overshoot.
Humans have skills and motivations (such as deception, manipulation and power-hungriness) which would be dangerous in AGIs. It seems plausible that the development of many of these traits was driven by competition with other humans, and that AGIs trained to answer questions or do other limited-scope tasks would be safer and less goal-directed. I briefly make this argument here.
Note that he claims that this may be true even if single/single alignment is solved, and all AGIs involved are aligned to their respective users.
It strikes me as interesting that much of the existing work that’s been done on multiagent training, such as it is, focusses on just examining the behaviour of artificial agents in social dilemmas. The thinking seems to be—and this was also suggested in ARCHES—that it’s useful just for exploratory purposes to try to characterise how and whether RL agents cooperate in social dilemmas, what mechanism designs and what agent designs promote what types of cooperation, and if there are any general trends in terms of what kinds of multiagent failures RL tends to fall into.
For example, it’s generally known that regular RL tends to fail to cooperate in social dilemmas, ‘Unfortunately, selfish MARL agents typically fail when faced with social dilemmas’. From ARCHES:
One approach to this research area is to continually ex-amine social dilemmas through the lens of whatever is the leading AI devel-opment paradigm in a given year or decade, and attempt to classify interest-ing behaviors as they emerge. This approach might be viewed as analogous to developing “transparency for multi-agent systems”: first develop inter-esting multi-agent systems, and then try to understand them.
There seems to be an implicit assumption here that something very important and unique to multiagent situations would be uncovered—by analogy to things like the flash crash. It’s not clear to me that we’ve examined the intersection of RL and social dilemmas enough to notice if this were true, if it were true, and I think that’s the major justification for working on this area.
One thing that you didn’t account for—the method of directly scaling the Rt by the multiple on the R0 (which seems to be around 1.55), is only a rough estimate of how much the Rt will increase by when the effective Rt is lowered in a particular situation. It could be almost arbitrarily wrong—intuitively, if the hairdressers are closed, that prevents 100% of transmission in hairdressers no matter how much higher the R0 of the virus is.
For this reason, the actual epidemiological models (there aren’t any for the US for the new variant, only some for the UK), have some more complicated way of predicting the effect of control measures. This from Imperial College:
We quantified the transmission advantage of the VOC relative to non-VOC lineages in twoways: as an additive increase in R that ranged between 0.4 and 0.7, and alternatively as amultiplicative increase in R that ranged between a 50% and 75% advantage. We were not ableto distinguish between these two approaches in goodness-of-fit, and either is plausiblemechanistically. A multiplicative transmission advantage would be expected if transmissibilityhad increased in all settings and individuals, while an additive advantage might reflect increasesin transmissibility in specific subpopulations or contexts.
The multiplicative ‘increased transmissibility’ estimate will therefore tend to underestimate the effect of control measures. The actual paper did some complicated Bayesian regression to try and figure out which model of Rt change worked best, and couldn’t figure it out.
Measures like ventilation, physical distancing when you do decide to meet up, and mask use will be more multiplicative in how the new variant diminishes their effect. The parts of the behaviour response that involve people just not deciding to meet up or do things in the first place, and anything involving mandatory closures of schools, bars etc. will be less multiplicative.
I believe this is borne out in the early data. Lockdown 1 in the UK took Rt down to 0.6. The naive ‘multiplicative’ estimate would say that’s sufficient for the new variant, Rt=0.93. The second lockdown took Rt down to 0.8, which would be totally insufficient. You’d need Rt for the old variant of covid down to 0.64 on the naive multiplicative estimate—almost what was achieved in March. I have a hard time believing it was anywhere near that low in the Tier 4 regions around Christmas.
But the data that’s come in so far seems to indicate that Tier 4 + Schools closed has either levelled off or caused slow declines in infections in those regions where they were applied.
First, the random infection survey—London and South East are in decline and East of England has levelled off (page 3). The UKs symptom study, which uses a totally different methodology, confirms some levelling off and declines in those regions—page 6. It’s early days, but clearly Rt is very near 1, and likely below 1 in London. The Financial Times cottoned on to this a few days late but no-one else seems to have noticed.
I think this indicates a bunch of things—mainly that infections caused by the new variant can and will be stabilized or even reduced by lockdown measures which people are willing to obey. It’s not impossible if it’s already happening.
To start, let’s also ignore phase shifts like overloading hospitals, and ignore fatigue on the hopes that vaccines coming soon will cancel it out, although there’s an argument that in practice some people do the opposite.
I agree with ignoring fatigue, but ignoring phase shifts? If it were me I’d model the entire control system response as a phase shift with the level for the switch in reactions set near the hospital overwhelm level—at least on the policy side, there seems to be an abrupt reaction specifically to the hospital overloading question. The British government pushed the panic button a few days ago in response to that and called a full national lockdown. I’d say the dominant contributor to control systems is something like a step function at a particular level near where hospitals are overwhelmed, and individual responses proportionate to exact levels of infection are a lesser part of it.
I think the model of the control system as a continuous response is wrong, and a phased all-or-nothing response for the government side of things, plus taking into account non-multiplicative effects on the Rt, would produce overall very different results—namely that a colossal overshoot of herd immunity in a mere few weeks is probably not happening. I note—outside view—that this is very similar to the predictive mistake made last Febuary/March with old Covid-19 - many around here were practically certain we were bound for an immediate (in a month or two) enormous herd immunity overshoot.
Many of the same thoughts were in my mind when I linked when I linked that study on the previous post.
----
IMO, it would help clarify arguments about the “control system” a lot to write down the ideas in some quantitative form.
...
This tells you nothing about the maximum power of my heating system. In colder temperatures, it’d need to work harder, and at some low enough temperature T, it wouldn’t be able to sustain 70F inside. But we can’t tell what that cutoff T is until we reach it. “The indoor temperature right now oscillates around 70F” doesn’t tell you anything about T.
I agree, and in fact the main point I was getting at with my initial comment is that in the two areas I talked about—namely the control system and the overall explanation for failure, there’s an unfortunate tendency to toss out quantitative arguments or even detailed models of the world and instead resort to intuitions and qualitative arguments—and then it has a tendency to turn into a referendum on your personal opinions about human nature and the human condition, which isn’t that useful for predicting anything. You can see this in how the predictions panned out—as was pointed out by some anonymous commenter, control system ‘running out of power’ arguments generally haven’t been that predictively accurate when it comes to these questions.
The rule-of-thumb that I’ve used—the Morituri Nolumus Mori effect—has fared somewhat better than the ‘control system will run out of steam sooner or later’ rule-of-thumb, both when I wrote that post and since. The MNM tends to predict last-minute myopic decisions that mostly avoid the worst outcomes, while the ‘out of steam’ explanation led people to predict that social distancing would mostly be over by now. But neither is a proper quantitative model.
In terms of actually giving some quantitative rigour to this question—it’s not easy. I made an effort in my old post, by saying how far a society can stray from a control system equilibrium is indicated by how low they managed to get Rt—but the ‘gold standard’ is to just work off model projections trained on already existing data like I tried to do.
As to the second question—overall explanation, there is some data to work off of, but not much. We know that preexisting measures of state capacity don’t predict covid response effectiveness, which along with other evidence suggests the ‘institutional schlerosis’ hypothesis I referred to in my original post. Once again, I think that a clear mechanism - ‘institutional sclerosis as part of the great stagnation’ - is a much better starting point for unravelling all this than the ‘simulacra levels are higher now’ perspective that I see a lot around here. That claim is too abstract to easily falsify or derive genuine in-advance predictions.
Everyone says the Culture novels are the best example of an AI utopia, but even though it’s a cliché to mention the culture, it’s a cliché for a good reason. Don’t start with Consider Phlebas (the first one), but otherwise just dive in. My other recommendation is the Commonwealth Saga by Peter F Hamilton and the later Void Trilogy—it’s not on the same level of writing quality as the Culture, although still a great story, but it depicts an arguably superior world to that of the Culture—with more unequivocal support of life extension and transhumanism.
The Commonwealth has effective immortality, a few downsides of it are even noticeable (their culture and politics is a bit more stagnant than we might like), but there’s never any doubt at all that it’s worth it, and it’s barely commented on in the story. The latter-day Void Trilogy Commonwealth is probably the closest a work of published fiction has come to depicting a true eudaemonic utopia that lacks the problems of the culture.