FWIW I’ve seen multiple double-mRNA-vaccinated people in my social circles who still got infected with delta (and in one case infected someone else who was double vaccinated). Two of the cases I know were symptomatic (but mild).
SoerenMind
Karma: 1,411
According to one expert, the immune system essentially makes bets on how often it will face a given virus and how the virus will mutate in the future:
https://science.sciencemag.org/content/372/6549/1392
By that logic, being challenged more often means that the immune system should have a stronger and longer-lasting response:
The immune system treats any new exposure—be it infection or vaccination—with a cost-benefit threat analysis for the magnitude of immunological memory to generate and maintain. There are resource-commitment decisions: more cells and more protein throughout the body, potentially for decades. Although all of the calculus involved in these immunological cost-benefit analyses is not understood, a long-standing rule of thumb is that repeated exposures are recognized as an increased threat. Hence the success of vaccine regimens split into two or three immunizations.
The response becomes even stronger when challenging the immune system with different versions of the virus, in particular a vaccine and the virus itself (same link).
Heightened response to repeated exposure is clearly at play in hybrid immunity, but it is not so simple, because the magnitude of the response to the second exposure (vaccination after infection) was much larger than after the second dose of vaccine in uninfected individuals. [...] Overall, hybrid immunity to SARS-CoV-2 appears to be impressively potent.
For SARS-CoV-2 this leads to a 25-100x stronger antibody response. It also comes with enhanced neutralizing breadth, and therefore likely some protection against future variants.
Based on this, the article above recommends combining different vaccine modalities such as mRNA (Pfizer, Moderna) and vector (AZ) (see also here).
Lastly, your question may be hard to answer without data, if we extrapolate from a similar question where the answer seems hard to predict in advance:
Additionally, the response to the second vaccine dose was minimal for previously infected persons, indicating an immunity plateau that is not simple to predict.
Suggestion for content 2: relationship to invariant causal prediction
Lots of people in ML these days seem excited about getting out of distribution generalization with techniques like invariant causal prediction. See e.g. this, this, section 5.2 here and related background. This literature seems promising but in discussions about inner alignment it’s missing. It seems useful to discuss how far it can go in helping solve inner alignment.
Suggestion for content 1: relationship to ordinary distribution shift problems
When I mention inner alignment to ML researchers, they often think of it as an ordinary problem of (covariate) distribution shift.My suggestion is to discuss if a solution to ordinary distribution shift is also a solution to inner alignment. E.g. an ‘ordinary’ robustness problem for imitation learning could be handled safely with an approach similar to Michael’s: maintain a posterior over hypotheses , with a sufficiently flexible hypothesis class, and ask for help whenever the model is uncertain about the output y for a new input x.
One interesting subtopic is whether inner alignment is an extra-ordinary robustness problem because it is adversarial: even the tiniest difference between train and test inputs might cause the model to misbehave. (See also this.)
Feedback on your disagreements with Michael:
I agree with “the consensus algorithm still gives inner optimizers control of when the system asks for more feedback”.
Most of your criticisms seem to be solvable by using a less naive strategy for active learning and inference, such as Bayesian Active Learning with Disagreement (BALD). Its main drawback is that exact posterior inference in deep learning is expensive since it requires integrating over a possibly infinite/continuous hypothesis space. But approximations exist.BALD (and similar methods) help with most criticisms:
It only needs one run, not 100. Instead, it samples hypotheses (let’s say 100) from a posterior .
It doesn’t suffer from dependence between runs because there’s only 1 run. It just has to take iid samples from its own posterior (many inference techniques do this).
It doesn’t require that the true hypothesis is always right. Instead each hypothesis defines a distribution over answers and it only gets ruled out when it puts 0% chance on the human’s answer. (For imitation learning, that should never happen)
It doesn’t require that one among the 100 hypotheses that is safe inputs. Drawback: It requires the weaker condition that input we encounter, one hypothesis (among 100) that is safe.
It converges faster because it actively searches for inputs where hypotheses disagree.
(Bayesian ML can even be adversarially robust with exact posterior inference.)
Apologies if I missed details from Michael’s paper.
Re 1) the codons, according to Christian Drosten, have precedence for evolving naturally in viruses. That could be because viruses evolve much faster than e.g. animals. Source: search for ‘codon’ and use translate here: https://www.ndr.de/nachrichten/info/92-Coronavirus-Update-Woher-stammt-das-Virus,podcastcoronavirus322.html
The link also has a bunch of content about the evolution of furin cleavage sites, from a leading expert.
Favoring China in the AI race
In a many-polar AI deployment scenario, a crucial challenge is to solve coordination problems between non-state actors: ensuring that companies don’t cut corners, monitoring them, just to name a few challenges. And in many ways, China is better than western countries at solving coordination problems within their borders. For example, they can use their authority over companies as these tend to be state-owned or owned by some fund that is owned by a fund that is state owned. Could this mean that, in a many-polar scenario, we should favor China in the race to build AGI?
Of course, the benefits of China-internal coordination may be outweighed by the disadvantages of Chinese leadership in AI. But these disadvantages seem smaller in a many-polar world because many actors, not just the Chinese government, share ownership of the future.
SoerenMind’s Shortform
Thanks—I agree there’s value to public peer review. Personally I’d go further than notifying authors and instead ask for permission. We already have a problem where many people (including notably highly accomplished authors) feel discouraged from posting due to the fear of losing reputation. Worse, your friends will actually read reviews of your work, unlike OpenReview. And I wouldn’t want to make this worse by implicitly making authors opt into a public peer review if that makes sense.
There are also some differences between forums and academia. Forums allow people to share unpolished work and see how the community reacts. I worry that highly visible public reviews may discourage some authors from posting this work, unless it’s obvious that they won’t get a highly visible negative review for their off-the-cuff thoughts without opting into it. Which seems doable within your (very useful) approach. I agree there’s a fine line here; just want to point out that not everyone is emotionally ready for this.
There’s also a strong chance that delta is the most transmissible variant we know even without its immune evasion (source: I work on this, don’t have a public source to share). I agree with your assessment that delta is a big deal.
This seems useful. But do you ask the authors for permission to review and give them an easy way out? Academic peer review is for good reasons usually non-public. The prospect of having one’s work reviewed in public seems likely to be extremely emotionally uncomfortable for some authors and may discourage them from writing.
Google seems to have solved some problem like the above for a multi-language-model (MUM):
”Say there’s really helpful information about Mt. Fuji written in Japanese; today, you probably won’t find it if you don’t search in Japanese. But MUM could transfer knowledge from sources across languages, and use those insights to find the most relevant results in your preferred language.”
Some reactions:
The Oxford/London nexus seems like a nice combination. It’s 38min by train between the two, plus getting to the stations (which in London can be a pain).
Re intellectual life “behind the walls of the colleges”: I haven’t perceived much intellectual life in my college, and much more outside. Maybe the part inside the colleges is for undergraduates?
I don’t have experience with long-range commuting into Oxford. But you can commute in 10-15 minutes by bike from the surrounding villages like Botley / Headington.
I don’t think anyone has mentioned Oxford, UK yet? It’s tiny. You could literally live on a farm here and still be 5-10 minutes from the city centre. And obviously it’s a realistic place for a rationalist hub. I haven’t perceived anti-tech sentiment here but haven’t paid attention either.
I agree that 1-3 need more attention, thanks for raising them.
Many AI scientists in the 1950s and 1960s incorrectly expected that cracking computer chess would automatically crack other tasks as well.
There’s a simple disconnect here between chess and self-supervised learning. You’re probably aware of it but it it’s worth mentioning. Chess algorithms were historically designed to win at chess. In contrast, the point of self-supervised learning is to extract representations that are useful in general. For example, to solve a new tasks we can feed the representations into a linear regression, another general algorithm. ML researchers have argued for ages that this should work and we already have plenty of evidence that it does.
How useful would it be to work on a problem where the LM “knows” can not be superhuman but it still knows how to do well and needs to be incentivized to do so? A currently prominent example problem is that LMs produce “toxic” content:
https://lilianweng.github.io/lil-log/2021/03/21/reducing-toxicity-in-language-models.html
Put differently, buying eggs only hurt hens via some indirect market effects, and I’m now offsetting my harm at that level before it turns into any actual harm to a hen.
I probably misunderstand but isn’t this also true about other offsetting schemes like convincing people to go vegetarian? They also lower demand.
Related, Acetylcholine has been hypothesized to signal to the rest of the brain that unfamiliar/uncertain things are about to happen
https://www.sciencedirect.com/science/article/pii/S0896627305003624
http://www.gatsby.ucl.ac.uk/~dayan/papers/yud2002.pdf
FWIW I wouldn’t read much into it if LMs were outperforming humans at next-word-prediction. You can improve on it by having superhuman memory and doing things like analyzing the author’s vocabulary. I may misremember but I thought we’ve already outperformed humans on some LM dataset?
Some standard ones like masks, but not at all times. They probably were in close or indoor contact with infected people without precautions.