On (3), I’m more saying, capabilities won’t be bottlenecked on more data or compute. Before, say, 2019 (GPT2 release) AI researchers weren’t actually using enough data or compute for many potential algorithmic innovations to be relevant, regardless of what was available theoretically at the time.
But now that we’re past a minimum threshold of using enough compute and data where lots of things have started working at all, I claim / predict that capabilities researchers will always be able to make meaningful and practical advances just by improving algorithms. More compute and more data could also be helpful, but I consider that to be kind of trivial—you can always get better performance from a Go or Chess engine by letting it run for longer to search deeper in the game tree by brute force.
And it’s had an unimaginably vast amount of “compute” in physical systems to work with over time.
A few billion years of very wasteful and inefficient trial-and-error by gradual mutation on a single planet doesn’t seem too vast, in the grand scheme of things. Most of the important stuff (in terms of getting to human-level intelligence) probably happened in the last few million years. Maybe it takes planet-scale or even solar-system scale supercomputers running for a few years to reproduce / simulate. I would bet that it doesn’t take anything galaxy-scale.
On (9): yeah, I was mainly just pointing out a potentially non-obvious use and purpose of some research that people sometimes don’t see the relevance of. Kind of straw, but I think that some people look at e.g. logical decision theory, and say “how the heck am I supposed to build this into an ML model? I can’t, therefore this is not relevant.”
And one reply is that you don’t build it in directly: a smart enough AI system will hit on LDT (or something better) all by itself. We thus want to understand LDT (and other problems in agent foundations) so that we can get out in front of that and see it coming.
On (3), I’m more saying, capabilities won’t be bottlenecked on more data or compute. Before, say, 2019 (GPT2 release) AI researchers weren’t actually using enough data or compute for many potential algorithmic innovations to be relevant, regardless of what was available theoretically at the time.
But now that we’re past a minimum threshold of using enough compute and data where lots of things have started working at all, I claim / predict that capabilities researchers will always be able to make meaningful and practical advances just by improving algorithms. More compute and more data could also be helpful, but I consider that to be kind of trivial—you can always get better performance from a Go or Chess engine by letting it run for longer to search deeper in the game tree by brute force.
A few billion years of very wasteful and inefficient trial-and-error by gradual mutation on a single planet doesn’t seem too vast, in the grand scheme of things. Most of the important stuff (in terms of getting to human-level intelligence) probably happened in the last few million years. Maybe it takes planet-scale or even solar-system scale supercomputers running for a few years to reproduce / simulate. I would bet that it doesn’t take anything galaxy-scale.
On (9): yeah, I was mainly just pointing out a potentially non-obvious use and purpose of some research that people sometimes don’t see the relevance of. Kind of straw, but I think that some people look at e.g. logical decision theory, and say “how the heck am I supposed to build this into an ML model? I can’t, therefore this is not relevant.”
And one reply is that you don’t build it in directly: a smart enough AI system will hit on LDT (or something better) all by itself. We thus want to understand LDT (and other problems in agent foundations) so that we can get out in front of that and see it coming.