gwern
Rohin’s opinion: I really liked the experiment demonstrating misalignment, as it seems like it accurately captures the aspects that we expect to see with existentially risky misaligned AI systems: they will “know” how to do the thing we want, they simply won’t be “motivated” to actually do it.
In the end humanity was saved by adding “Super safe.” to all their requests of the AGI
My counter joke (in EAI) was:
“AGI but its supr safe.”
(GPT-3 is an agent-predicting agent.)
Might be good to revisit this idea in light of NFTs.
Has anyone pointed out to Chollet yet that scaling up bottle rockets is a great way to go to space?
That’s impressive, considering D-K isn’t real.
It’s based on those estimates and the systematic biases in such methods & literatures. Just as you know that psychology and medical effects are always overestimated and can be rounded down by 50% to get a more plausible real world estimate, such information-theoretic methods will always overestimate model performance and underestimate human performance, and are based on various idealizations: they use limited genres and writing styles (formal, omitting informal like slang), don’t involve extensive human calibration or training like the models get, don’t involve any adversarial examples, don’t try to test human reasoning by writing up texts made up of logical riddles and puzzles or complicated cause-and-effect scenarios or even things like Winograd Schemas, are time-biased, etc. We’ve seen a lot of these issues come up in benchmarking, like ImageNet models outside ImageNet despite hitting human parity or superiority. (If we are interested in truly testing ‘compression = intelligence’, we need texts which stress all capabilities and remove all of those issues.)
So given Shannon’s interval’s lower end is 0.6, and Grassberger’s asymptotic is 0.8 (the footnote 11) and a widespread of upper bounds going down to 1.3 along with extremely dumb fast algorithms hitting 2, I am comfortable with rounding them downish to get estimates of 0.7 bpc being the human performance; and I expect that to, if anything, be still underestimating true human peak performance, so I wouldn’t be shocked if it was actually more like 0.6 bpc.
And some of the past QC claims for ML have not panned out. Like, I think there was a Quantum Monte Carlo claimed to be potentially useful for ML which could be done on cheaper QC archs, but then it turned out to be doable classically...? In any case, I have been reading about QCs all my life, and they have yet to become relevant to anything I care about; and I assume Scott Aaronson will alert us should they suddenly become relevant to AI/ML/DL, so the rest of us should go about our lives until that day.
The obvious difference is that the second does not include Elgazzar, while the first includes Elgazzar, which is bad for the first one because Elgazzar faked its data so incompetently it has been retracted: https://grftr.news/why-was-a-major-study-on-ivermectin-for-covid-19-just-retracted/ https://gidmk.medium.com/is-ivermectin-for-covid-19-based-on-fraudulent-research-5cc079278602 https://www.theguardian.com/science/2021/jul/16/huge-study-supporting-ivermectin-as-covid-treatment-withdrawn-over-ethical-concerns
The previous AF2 discussions were largely a waste of space because the little abstract they had to provide for CASP14 provided hardly anything to go on. But now we have not just a full writeup but source code and models too! Now I consider it worth discussing.
AlphaFold 2 paper released: “Highly accurate protein structure prediction with AlphaFold”, Jumper et al 2021
Moore: yes.
QC: AFAIK it’s irrelevant?
GPT-3: it used up a particular kind of overhang you might call the “small-scale industrial CS R&D budget hardware overhang”. (It would certainly be possible to make much greater than GPT-3-level progress, but you’d need vastly larger budgets: say, 10% of a failed erectile-dysfunction drug candidate, or 0.1% of the money it takes to run a failed European fusion reactor or particle collider.) So, I continue to stand by my scaling hypothesis essay’s paradigm that as expected, we saw some imitation and catchup, but no one created a model much bigger than GPT-3, never mind one that was >100x bigger the way GPT-3 was to GPT-2-1.5b, because no one at relevant corporations truly believes in scaling or wishes to commit the necessary resources, or feels that it’s near a crunchtime where there might be a rush to train a model at the edge of the possible, and OA itself has been resting on its laurels as it turns into a SaaS startup. (We’ll see what the Anthropic refugees choose to do with their $124m seed capital, but so far they appear to be making a relaxed start of it as well.)
The overhang GPT-3 used up should not be confused with other overhangs. There are many other hardware overhangs of interest: the hardware overhang of the experience curve where the cost halves every year or two; the hardware overhang of a distilled/compressed/sparsified model; the hardware overhang of the global compute infrastructure available to a rogue agent. The small-scale industrial R&D overhang is the relevant and binding one… for now. But the others become relevant later on, under different circumstances, and many of them keep getting bigger.
Given the timing of Jensen’s remarks about expecting trillion+ models and the subsequent MoEs of Switch & Wudao (1.2t) and embedding-heavy models like DLRM (12t), with dense models still stuck at GPT-3 scale, I’m now sure that he was referring to MoEs/embeddings, so a 100t MoE/embedding is both plausible and also not terribly interesting. (I’m sure Facebook would love to scale up DLRM another 10x and have embeddings for every SKU and Internet user and URL and video and book and song in the world, that sort of thing, but it will mean relatively little for AI capabilities or risk.) After all, he never said they were dense models, and the source in question is marketing, which can be assumed to accentuate the positive.
More generally, it is well past time to drop discussion of parameters, and switch to compute-only as we can create models with more parameters than we can train (you can fit a 100t-param with ZeRo into your cluster? great! how you gonna train it? Just leave it running for the next decade or two?) and we have no shortage of Internet data either: compute, compute, compute! It’ll only get worse if some new architecture with fast weights comes into play, and we have to start counting runtime-generated parameters as ‘parameters’ too. (eg Schmidhuber back in like the ’00s showed off archs which used… Fourier transforms? to have thousands of weights generate hundreds of thousands of weights or something. Think stuff like hypernetworks. ‘Parameter’ will mean even less than it does now.)
Using CLIP is a pretty weird way to go. It’s like using a CNN classifier to generate images: it can be done, but like a dog walking, we’re more surprised to see it work at all. If you think about how a contrastive loss works, it’s less surprising why CLIP-guided images look the way they do, and do things like try to repeat an object many time: if you have a prompt like “Mickey Mouse”, what could be even more Mickey-Mouse-y than MM tiled a dozen times? That surely maximizes its embedding encoding ‘Mickey Mouse’, and its distance from non-Disney-related image embeddings like “Empire State Building”! Whether you can really induce sharp coherent images from any scaled-up CLIP like ALIGN is unclear: contrastive losses just might not learn these things, and the generative model needs to do all the work. (CLIP would still remain useful as an extremely handy way to take any image generative model and quickly turn it into an text-editable model, or possible text->image model. One could probably do better by an explicit approach like DALL-E, but CLIP is available now and DALL-E is not.)
If you are interested in SOTA image generation quality rather than weird CLIP hacks, you should look at:
“VQGAN: Taming Transformers for High-Resolution Image Synthesis”, Esser et al 2020
“SR3: Image Super-Resolution via Iterative Refinement”, Saharia et al 2021 / “Diffusion Models Beat GANs on Image Synthesis”, Dhariwal & Nichol 2021
“Alias-Free GAN (Generative Adversarial Networks)”, Karras et al 2021
If you want to zero out mutational load and create a modal genome, you’re approximately 2 orders of magnitude off in the number of edits you need to do. (The number of evolutionarily-conserved protein-coding regions has little to do with the number of total variants across the billions of basepairs in a specific human’s genome.) Considering that it is unlikely that we will get base editors, ever, which have total error rates approaching 1 in millions, one will have to be a little more clever about how one goes about it. (Maybe some sort of multi-generational mass mutagenesis-editing / screening loop?)
Anyway, I would point out that you can do genome synthesis on a much cheaper scale than whole-genome: whole-chromosome is an obvious intermediate point which would be convenient to swap out. And for polygenic traits, optimizing a single chromosome might push the phenotype out as far as you want to go in a single generation anyway.
Has anyone done this, anywhere?
https://arxiv.org/abs/1908.04734#deepmind https://www.lesswrong.com/posts/pjzhmtivXd8zgKXDT/designing-agent-incentives-to-avoid-reward-tampering https://www.lesswrong.com/posts/kxPiL4zNSPR249wsC/an-114-theory-inspired-safety-solutions-for-powerful https://medium.com/@deepmindsafetyresearch/designing-agent-incentives-to-avoid-reward-tampering-4380c1bb6cd
On a related note, one of the neater papers I’ve seen recently:
“Air Pollution and Adult Cognition: Evidence from Brain Training”, La Nauze & Severnini 2021: using 4.6m scores from Lumosity brain-game players to measure PM2.5 cognitive effects: large but heterogeneous by task/age/experience—this may explain the inconsistencies of all the previous air-pollution/cognition studies.
He definitely didn’t mention it much (which is part of what gave me that impression—in general, Sam’s public output is always very light about the details of OA research). I dunno about deleting. Twitter search is terrible; I long ago switched to searching my exported profile dump when I need to refind an old tweet of mine.
The statistical silver bullet you’re looking for, assuming you can’t run multi-decade RCTs or convince entire countries to dramatically change their water policies (sounds a bit tricky?), is variance components analysis.
You want a variance component analysis equivalent to heritability, but for water/obesity. This is a perfect match: you have a very high-dimensional measurement (GC-MS of drinking water & foodstuffs), whose components you can’t observe because you don’t even know how many there are much less which ones (like the unknown chemicals that the original contaminants may react into along the way**), where direct prediction/regression would fail because it requires absurd amounts of data (especially given the statistical issues they outline), and where your original research question is not “where is the needle in the haystack” but “does this haystack have any needles at all”.
In behavioral genetics, if you plot pairs of people by their genetic similarity and their phenotypic similarity, you will find that people similar on both tend to cluster; and the more heritable the trait is, the more they cluster. You can do this for kinds of similarity which aren’t ‘genetic’. The idea of variance components is to switch from directly correlating/regressing the specific chemical levels in water, trying to measure the exact direct effect of each chemical on obesity, to instead asking, “how similar, in terms of overall water chemical composition, are similarly far or thin people? Do the chemical spectrums of fat people tend to look similar, and the spectrums of thin people look similar? Do they co-vary, and cluster?”
Think of the water GC-MS as a sort of fingerprint or high-dimensional summary. It doesn’t need to contain the actual critical datapoint, just a lot of correlates/proxies thereof (eg you can just shine some light on leaves and it’ll work!). The point is not to directly measure the culprit, but to indirectly measure how distant the datapoints are; perhaps chemical A shows up and chemical B does not show up clearly, and B is the real cause, but as long as chemical A correlates with chemical B (maybe it’s a different reaction product, or just gets manufactured or used in close correlation with B), then you will see that fatties tend to have close-together water spectra (because of A) and you will have strong evidence that “there is something in the water” because where the bad water clusters the fatties cluster.
So, you’d do something like record the BMI of 10,000 people, take a sample of their drinking water, GC-MS it (ideally, but many other analytical techniques could be used and maybe some would be better here, I’ll defer to any actual analytical chemists on that topic), and then plug into a mixed linear-model. You can plug in covariates as necessary (families/pedigrees to jointly model genetic heritability, ZIP codes, age, sex, local mining/factories, already known harmful contaminants the GC-MS detected, that sort of thing). This can be done at multiple levels: individuals nested in households, within regions, within countries, etc. The amount of covariance between the water spectrum and BMI measures the amount of variance the contents of the water en masse can explain, and depending on measurement error and whatnot, given the genetic heritability bounding how much water-heritability can explain, you should get somewhere <30%. Under most theories of obesity, it’s difficult to see why one would predict a water-heritability different from 0%*, especially if one has already included altitude/SES/household/region as covariates (which should cover all the obvious confounders like “maybe contaminated water is just a proxy for living in a poor region / having mining nearby / being poor”), so anything >0% is highly suspicious.
This doesn’t require any RCTs (or interventions of any kind), longitudinal datasets, n in the millions, or any of the barriers to most of the proposed tests. You only need a cross-section of obesity/water-samples. Individual level is ideal, but doing it with geographic units can be useful too if that is what is available across enough units (individual n are much nosier than k, but also much easier to get high statistical power with). This may even already exist! Drinking water is often tested for many reasons, and obesity health data is standard medical data collected in almost any relevant research; so there ought to be a lot of possible cross-references.
* you could do this with food as well but I focus on water because water is a lot more consistent day to day, and food results would be ambiguous. A single day’s food is still not very representative of individual consumption (maybe your wife cooks healthy at home but then you eat processed garbage at the office, and the sample comes from the weekend), so it provides a very poor measurement of the overall diet’s chemical signature. And if you demonstrated a large food-heritability of obesity, this would just be taken as proving that “see? sugar / fat / calories / highly-palatable processing causes everything, just like we said all along!”, and be weak evidence for contamination at best.
** They also raise the problem of interactions & nonlinear responses. These are issues in behavioral genetics too, of course, with dominance/epistasis particularly, and can be dealt with similarly in the water context: “narrowsense” vs “broadsense” heritability covers simple additive vs interactions, and you can do things like DeFries-Fulker, I think, for nonlinear responses. My guess is that contamination would be ‘additive’ mostly, for similar underlying reasons as genetics tends to be ‘additive’: many interactions are just constants because they are done on a fixed background of chemistry, and promiscuous interactions average out to additive effects. But this is the sort of thing to worry about only after having done the straightforward work.