Fundamentals of Skill, Welford (1968)
I’ve uploaded a scan if you want to look.
gwern
Karma: 42,603
“Designing agent incentives to avoid reward tampering”, DeepMind
Is Net Smart very practical? The introduction sounds more theoretical and generic, and it’s a good 7 years old now. (I noticed when I saw references in that link to long-defunct websites like CureTogether.)
In theory? You just generate a few random samples with the current text as the prefix and display them. In practice, there’s already tools to do this: Talk to Transformer does autocomplete. Even better, IMO, is Deep TabNine for programming languages, trained off Github.
The impression I got from asking people like James about metformin side-effects when I was trying a cost-benefit is that most of it has quick onset, like the gastrointestinal distress, and if you can’t fix it by modifying the dose, you can simply discontinue it ie you have option value. This would reduce the EV a little but is not that big a deal. After all, metformin is one of the most (the most?) widely used chronic prescription drugs in the world & regarded as very safe, so the side effects can’t be that bad, one would think.
The question of redundancy with other interventions is a more concerning one. Not all the metformin papers are positive in this regard. Here’s a small paper suggesting that metformin blunts the benefits of exercise, and “Metformin alters the gut microbiome of individuals with treatment-naive type 2 diabetes, contributing to the therapeutic effects of the drug”, Wu et al 2017, suggests part of metformin’s benefits is by changing the microbiome, but of course, exercise or diet or lifestyle changes might also be changing the microbiome in precisely the same way… For diabetics, who have done what little they are able or willing to do, that presumably is not happening enough to cure their diabetes and so the average metformin effect is still worthwhile, but for those more rigorous about longevity, who knows?
I have similar concerns about baby aspirin and everything postulated to involve inflammation, and perhaps also the senolytics as well: they often seem to be hypothesized to be acting through similar pathways (eg inflammation causes/is caused by senescent cells, some say, but if exercise kills senescent cells by inducing autophagy, doesn’t that imply it’d be at least partially redundant with taking a senolytic drug?). I’m not sure what could be done here except to directly test the potential for interactions in factorial experiments.
July 2019 gwern.net newsletter
Citations can be used as the metadata. One of the closest corresponding things in cliometrics are ‘sleeping beauty’ papers, which instead of the usual gradual decline in citation rate, suddenly see a big uptick many years afterwards. The recent ‘big teams vs small teams’ paper discussed sleeping beauty papers a little: https://www.gwern.net/docs/statistics/bias/2019-wu.pdf You could also take multiple discovery as quantifying repetition, since one of the most common ways for a multiple to happen is for it to happen in a different field where it is also important/useful but they haven’t heard of the original discovery in the first field.
There’s a nice version of this with Ed Boyden on how old papers helped lead to the hot new ‘expansion microscopy’ thing (funded, incidentally, by OpenPhil): https://medium.com/conversations-with-tyler/tyler-cowen-ed-boyden-neuroscience-3907eccbd4ca
As a historical fact, you certainly can invent selective breeding without knowing anything we would consider true: consider Robert Bakewell and the wildly wrong theories of heredity current when he invented line breeding and thus demonstrated that breeds could be created by artificial selection. (It’s unclear what Bakewell and/or his father thought genetics was, but at least in practice, he seems to have acted similarly to modern breeding practices in selecting equally on mothers/fathers, taking careful measurements and taking into account offspring performance, preserving samples for long-term comparison, and improving the environment as much as possible to allow maximum potential to be reached.) More broadly, humans had no idea what they were doing when they were domesticated everything; if Richard Dawkins is to be trusted, it seems that the folk genetics belief was that traits are not inherited and everything regressed to an environmental mean, and so one might as well eat one’s best plants/animals since it’ll make no difference. And even more broadly, evolution has no idea what ‘it’ is doing for anything, of course.
The problem is, as Eliezer always pointed out, that selection is extremely slow and inefficient compared to design—the stupidest possible optimization process that’ll still work within the lifetime of Earth—and comes with zero guarantees of any kind. Genetic drift might push harmful variants up, environmental fluctuations might extinguish lineages, reproductively fit changes which Goodhart the fitness function might spread, nothing stops a ‘treacherous turn’, evolved systems tend to have minimal modularity and are incomprehensible, evolution will tend to build in instrumental drives which are extremely dangerous if there is any alignment problem (which there will be), sexual selection can drive a species extinct, evolved replicators can be hijacked by replicators on higher levels like memetics, any effective AGI design process will need to learn inner optimizers/mesa-optimizers which will themselves be unpredictable and only weakly constrained by selection, and so on. If there’s one thing that evolutionary computing teaches, it’s that these are treacherous little buggers indeed (Lehman et al 2018). The optimization process gives you what you ask for, not what you wanted.
So, you probably can ‘evolve’ an AGI, given sufficient computing power. Indeed, considering how many things in DL or DRL right now take the form of ‘we tried a whole bunch of things and X is what worked’ (note that a lot of papers are misleading about how many things they tried, and tell little theoretical stories about why their final X worked, which are purely post hoc) and only much later do any theoreticians manage to explain why it (might) work, arguably that’s how AI is proceeding right now. Things like doing population-based training for AlphaStar or NAS to invent EfficientNet are just conceding the obvious and replacing ‘grad student descent’ with gradient descent.
The problem is, we won’t understand why they work, won’t have any guarantees that they will be Friendly, and they almost certainly will have serious blindspots/flaws (like adversarial examples or AlphaGo’s ‘delusions’ or how OA5/AlphaStar fell apart when they began losing despite playing apparently at pro level before). NNs don’t know what they don’t know, and neither do we.
Nor are these flaws easy to fix with just some more tinkering. Much like computer security, you can’t simply patch your way around all the problems with software written in C (as several decades of endless CVEs has taught us); you need to throw it out and start with formal methods to make errors like buffer overflows impossible. Adversarial examples, for instance: I recall that one conference had something like 5 adversarial defenses, all defined heuristically without proof of efficacy, and all of them were broken between the time of submission and the actual conference. Or AlphaGo’s delusions couldn’t be fixed despite quite elaborate methods being used to produce Master (which at least had better ELO) until they switched to the rather different architecture of AlphaZero. Neither OA5 nor AlphaStar has been convincingly fixed that I know of, they simply got better to the point where human players couldn’t exploit them without a lot of practice to find reproducible ways of triggering blindspots.
So, that’s why you want all the math. So you can come up with provably Friendly architectures without hidden flaws which simply haven’t been triggered yet.
“From Statistical Significance To Effect Estimation: Statistical Reform In Psychology, Medicine And Ecology”, Fidler 2005; a broad but still in depth thesis on the history of NHST and attempts to reform it.
Does the abstract not work for you?
How Should We Critique Research? A Decision Perspective
Avalon is another example, with better current performance.
Tomasello’s work stresses mindreading, in particular the ability for humans to carry joint attention [link].
Link seems to be missing.
I didn’t say anything about chess or shogi because I don’t recall any ablation for A0, I just remember the one in the AG0 paper for Go. The AG0 is definitely at or close to professional level and better than ‘good amateur’. And I would consider a non-distributed PUCT with no rollouts or other refinements to be a ‘simple tree search’: it doesn’t do any rollouts, and the depth is seriously limited by running on only a single machine w/4 TPUs with a few seconds for search: as the AG0 paper puts it, “Finally, it uses a simpler tree search that relies upon this single neural network to evaluate positions and sample moves, without performing any Monte-Carlo rollouts...we chose to use the simplest possible search algorithm”.
June 2019 gwern.net newsletter
Sure. But the final player does not use MCTS, and it’s interesting that it’s not necessary then. (It’s even more interesting that the way they discovered they didn’t need MCTS is by hyperparameter optimization, but that’s a different discussion.)
You want the ‘AlphaGo Zero’ paper, not the ‘AlphaZero’ papers, which merely simplify it and reuse it in other domains; the AGZ paper is more informative than the AZ papers. See Figure 6b, and pg25 for the tree search
Figure 6b shows the performance of each program on an Elo scale. The raw neural network, without using any lookahead, achieved an Elo rating of 3,055. AlphaGo Zero achieved a rating of 5,185, compared to 4,858 for AlphaGo Master, 3,739 for AlphaGo Lee and 3,144 for AlphaGo Fan.
So the raw NN, a single pass and selecting the max, is 3k ELO, about 100 ELO under AlphaGo Fan, which soundly defeated a human professional (Fan Hui). I’m not sure whether −100 ELO is enough to demote it to amateur status, but it’s at least clearly not that far from professional in the worst case.
The paper includes the ELO for just the NN. I believe it’s professional level but not superhuman, but you should check if you really need to know. However, note that Alphazero’s actual play doesn’t use MCTS at all, it uses a simple tree search which only descends a few ply.
I’ve summarized my problems with Mouse Utopia: https://www.gwern.net/Questions#mouse-utopia