Bit of a tangent, but while we might plausibly run out of cheap oil in the near future, the supply of expensive, unconventional oil is vast. By vast I mean ‘several trillion barrels of known reserves’, against an annual consumption of 30bn.
Question is just how much of those reserves are accessible at each price point. This is really hard to answer well, so instead here’s an anecdote that’ll stick in your head: recent prices ($50-$100/bbl) are sufficient that the US is now the largest producer of oil in the world, and a net exporter to boot.
For what it’s worth, this whole unconventional oil thing has appeared from nowhere the last ten years, and it’s been a shock to a lot of people.
Thanks for the feedback! I’ve cleaned up the constraints section a bit, though it’s still less coherent than the first section.
Out of curiosity, what was it that convinced you this isn’t an infohazard-like risk?
While you’re here and chatting about D.5 (assume you meant 5), another tiny thing that confuses me—Figure 21. Am I right in reading the bottom two lines as ‘seeing 255 tokens and predicting the 256th is exactly as difficult as seeing 1023 tokens and predicting the 1024th’?
e: Another look and I realise Fig 20 shows things much more clearly—never mind, things continue to get easier with token index.
Though it’s not mentioned in the paper, I feel like this could be because the scaling analysis was done on 1024-token sequences. Maybe longer sequences can go further.
It’s indeed strange no-one else has picked up on this, which makes me feel I’m misunderstanding something. The breakdown suggested in the scaling law does imply that this specific architecture doesn’t have much further to go. Whether the limitation is in something as fundamental as ‘the information content of language itself’, or if it’s a more-easily bypassed ‘the information content of 1024-token strings’, is unclear.
My instinct is for the latter, though again by the way no-one else has mentioned it—even the paper authors—I get the uncomfortable feeling I’m misunderstanding something. That said, being able to write that quote a few days ago and since have no-one pull me up on it has increased my confidence that it’s a viable interpretation.
‘Why the hell has our competitor got this transformative capability that we don’t?’ is not a hard thought to have, especially among tech executives. I would be very surprised if there wasn’t a running battle over long-term perspectives on AI in the C-suite of both Google Brain and DeepMind.
If you do want to think along these lines though, the bigger question for me is why OpenAI released the API now, and gave concrete warning of the transformative capabilities they intend to deploy in six? twelve? months’ time. ‘Why the hell has our competitor got this transformative capability that we don’t?’ is not a hard thought now, but it that’s largely because the API was a piece of compelling evidence thrust in all of our faces.
Maybe they didn’t expect it to latch into the dev-community consciousness like it has, or for it to be quite as compelling a piece of evidence as it’s turned out to be. Maybe it just seemed like a cool thing to do and in-line with their culture. Maybe it’s an investor demo for how things will be monetised in future, which will enable the $10bn punt they need to keep abreast of Google.
hey man wanna watch this language model drive my car
Thinking about this a bit more, do you have any insight on Tesla? I can believe that it’s outside DM and GB’s culture to run with the scaling hypothesis, but watching Karpathy’s presentations (which I think is the only public information on their AI program?) I get the sense they’re well beyond $10m/run by now. Considering that self-driving is still not there—and once upon a time I’d have expected driving to be easier than Harry Potter parodies—it suggests that language is special in some way. Information density? Rich, diff’able reward signal?
I’d say it’s at least 30% likely that’s the case! But if you believe that, you’d be pants-on-head loony not to drop a billion on the ‘residual’ 70% chance that you’ll be first to market on a world-changing trillion-dollar technology. VCs would sacrifice their firstborn for that kind of deal.
Entirely seriously: I can never decide whether the drunkard’s search is a parable about the wisdom in looking under the streetlight, or the wisdom of hunting around in the dark.
There’s a LW thread with a collection of examples, and there’s the beta website itself.
Feels worth pasting in this other comment of yours from last week, which dovetails well with this:
DL so far has been easy to predict—if you bought into a specific theory of connectionism & scaling espoused by Schmidhuber, Moravec, Sutskever, and a few others, as I point out in https://www.gwern.net/newsletter/2019/13#what-progress & https://www.gwern.net/newsletter/2020/05#gpt-3 . Even the dates are more or less correct! The really surprising thing is that that particular extreme fringe lunatic theory turned out to be correct. So the question is, was everyone else wrong for the right reasons (similar to the Greeks dismissing heliocentrism for excellent reasons yet still being wrong), or wrong for the wrong reasons, and why, and how can we prevent that from happening again and spending the next decade being surprised in potentially very bad ways?
Personally, these two comments have kicked me into thinking about theories of AI in the same context as also-ran theories of physics like vortex atoms or the Great Debate. It really is striking how long one person with a major prior success to their name can push for a theory when the evidence is being stacked against it.
A bit closer to home than DM and GB, it also feels like a lot of AI safety people have missed the mark. It’s hard for me to criticise too loudly because, well, ‘AI anxiety’ doesn’t show up in my diary until June 3rd (and that’s with a link to your May newsletter). But a lot of AI safety work increasingly looks like it’d help make a hypothetical kind of AI safe, rather than helping with the prosaic ones we’re actually building.
I’m committing something like the peso problem here in that lots of safety work was—is—influenced by worries about the worst-case world, where something self-improving bootstraps itself out of something entirely innocuous. In that sense we’re kind of fortunate that we’ve ended up with a bloody language model fire-alarm of all things, but I can’t claim that helps me sleep at night.
GPT-3 does indeed only depend on the past few thousand words. AI Dungeon, however, can depend on a whole lot more.
Be careful using AI Dungeon’s behaviour to infer GPT-3′s behaviour. I am fairly confident that Latitude wrap your Dungeon input before submitting it to GPT-3; if you put in the prompt all at once, that’ll make for different model input than putting it in one line at a time.
I am also unsure as to whether the undo/redo system sends the same input to the model each time. Might be Latitude adds something to encourage an output different to the ones you’ve already seen.
Alternately phrased: much of the observed path dependence in this instance might be in Dragon, not GPT-3.
I think that going forward there’ll be a spectrum of interfaces to natural language models. At one end you’ll have fine-tuning, and at the other you’ll have prompts. The advantage of fine-tuning is that you can actually apply an optimizer to the task! The advantage of prompts is anyone can use them.
In the middle of the spectrum, two things I expect are domain-specific tunings and intermediary models. By ‘intermediary models’ I mean NLP models fine-tuned to take a human prompt over a specific area and return a more useful prompt for another model, or a set of activations or biases that prime the other model for further prompting.
The ‘specific area’ could be as general as ‘less flights of fancy please’.