Engineer at CoinList.co. Donor to LW 2.0.
ESRogs
Karma: 4,285
Claim 4: GPT-N need not be “trying” to predict the next word. To elaborate: one model of GPT-N is that it is building a world model and making plans in the world model such that it predicts the next word as accurately as possible. This model is fine on-distribution but incorrect off-distribution. In particular, it predicts that GPT-N would e.g. deliberately convince humans to become more predictable so it can do better on future next-word predictions; this model prediction is probably wrong.
I got a bit confused by this section, I think because the word “model” is being used in two different ways, neither of which is in the sense of “machine learning model”.
Paraphrasing what I think is being said:
An observer (us) has a model_1 of what GPT-N is doing.
According to their model_1, GPT-N is building its own world model_2, that it uses to plan its actions.
The observer’s model_1 makes good predictions about GPT-N’s behavior when GPT-N (the machine learning model_3) is tested on data that comes from the training distribution, but bad predictions about what GPT-N will do when tested (or used) on data that does not come from the training distribution.
The way that the observer’s model_1 will be wrong is not that it will be fooled by GPT-N taking a treacherous turn, but rather the opposite—the observer’s model_1 will predict a treacherous turn, but instead GPT-N will go on filling in missing words, as in training (or something else?).
Is that right?
Let , where and
[...] The second rule says that is orthogonal to itself
Should that be “is not orthogonal to itself”? I thought the meant non-orthogonal, so would think means that is not orthogonal to itself.
(The transcript accurately reflects what was said in the talk, but I’m asking whether Scott misspoke.)
But once you let it do more computation, then it doesn’t have to know anything at all, right? Like, maybe the best go bot is, “Train an AlphaZero-like algorithm for a million years, and then use it to play.”
I know more about go than that bot starts out knowing, but less than it will know after it does computation.
I wonder if, when you use the word “know”, you mean some kind of distilled, compressed, easily explained knowledge?
You have to be able to know literally everything that the best go bot that you have access to knows about go.
In your mind, is this well-defined? Or are you thinking of a major part of the challenge as being to operationalize what this means?
(I don’t know what it means.)
(I don’t expect to live on or immediately next to the proto-campus, but it would be cool to be somewhat nearby.)
I am expecting to “settle down” in either the Bay Area or Seattle. So I like the Bellingham option.
Thanks!
Here, there’s minimal dependence on ACT, but a negative dependence on , meaning that extreme ACT scores (high or low) both lead to lower likely-to-graduate scores.
Does that seem counterintuitive to you? Remember, we are taking a student who is already enrolled in a particular known college and predicting how likely that are to graduate from that college.
Sounds like a classic example of Simpson’s paradox, no?
My current theory for what happened is that everyone bought into this delusion about the value of bitcoin, but that unlike other bubbles it didn’t burst because Bitcoin has a limited supply and there is literally nothing to anchor its value. So there’s no point where investors give up and sell because there is literally no point at which it’s overpriced.
This actually sounds pretty close to what you might call the “bubble theory of money”: that money is a bubble that doesn’t pop, that certain (relatively) useless commodities can become money if enough people think of them that way, and when that happens their price is inflated, relative to their use value.
This isn’t something that will happen to every commodity. Whether it happens depends both on the properties of the commodity, and also on things like memes and Schelling points.
Bitcoin has enough useful properties (it’s like gold, but digital), and, because of its first-mover advantage, is the Schelling point for digital store-of-value (not that it couldn’t be replaced, but it’s a very up-hill battle), so it has become money, in this sense.
(On the memes-and-Schelling-points thing, see also: The Most Important Scarce Resource is Legitimacy, by Vitalik Buterin.)
the first 5-12 million dollar tab
You mean GPT-3? Are you asking whether it’s made enough money to pay for itself yet?
I believe that you (and the Twitter thread) are saying something meaningful, but I’m having trouble parsing it.
I had thought of the difference between variance and volatility as just that one is the square of the other. So saying that the VIX is “variance in vol units, but not volatility” doesn’t mean anything to me.
I think these are the critical tweets:
VIX is an index that measures the market implied level of 1-month variance on the S&P 500, or the square root thereof (to put it back in units we are used to).
This is not the same as volatility. A variance swap’s payoff is proportional to volatility squared. If you are short a variance swap at 10%, and then realized volatility turns out to be 40%, you lose your notional vega exposure times 16 (= 40^2 / 10^2 ).
To compensate for this, an equity index variance swap level is usually 2-3 points above the corresponding at the money implied volatility. So don’t look at VIX versus realized vol and make statements about risk premium without recognizing this extreme tail risk.
I was with him at “a variance swap’s payoff is proportional to volatility squared”. That matches my understanding of volatility as the square root of variance. But then I don’t get the next point about realized volatility needing to be “compensated for”.
Anybody care to explain?
Thanks!
Which all make them even easier targets for criticism, and make confident enthusiasm for an idea increasingly correlated with being some kind of arrogant fool.
But it also means conviction is undervalued, and it might be a good time to buy low!
I hold positions in Bitcoin, Ethereum, and Tesla through Exchange Traded Funds.
For Bitcoin and Ether, do you mean the Grayscale trusts, GBTC and ETHE? My impression is that these are similar to ETFs, but not exactly the same thing, and I’m not aware of other ETFs that give you exposure to crypto (except for the small amount of exposure you’d get from owning shares in companies that have a little BTC on their balance sheet, like Tesla, Square, or MicroStrategy).
The difference between a TSAR bomb (or its modern equivalent) and the lowest settings of a mini-nuke is still an order of magnitude larger than the difference between the conventional “mother of all bombs” and a hand grenade. The Beirut explosion last year was the size of the hand grenade blast in this analogy
I didn’t quite understand the last sentence here. Are you saying A) that the Beirut explosion was about the same size as a mini-nuke blast would be, or that B) MOAB : hand grenade :: TSAR bomb : Beirut explosion? (In which case the Beirut explosion would be larger than a mini-nuke explosion, if your claim about relative differences in the first sentence is correct.)
In other words, I take the first part of what you wrote to be saying that (TSAR bomb / mini-nuke) > (MOAB / grenade), but then I’m not sure whether the second part is saying that A) (TSAR bomb / Beirut explosion) = (TSAR bomb / mini-nuke), or B) (TSAR bomb / Beirut explosion) = (MOAB / grenade).
Is one of either A or B correct? (Or did you mean something else entirely?)
sometimes people think of things as being either X or Y, and then learn an argument for why this dichotomy doesn’t make sense. As a result, they might reject the dichotomy entirely
This reminds me of the Fallacy of Gray.
I’m definitely left wondering what AI Alignment research is left at OpenAI
You may be interested to know that Jan Leike recently joined OpenAI and will lead their alignment team.
Suppose you want to bet on interest rates rising—would buying value stocks and shorting growth stocks be a good way to do it? (With the idea being that, if rates rise, future earnings will be discounted more and present earnings valued relatively more highly.)
And separately from whether long-value-short-growth would work, is there a more canonical or better way to bet on rates rising?
Just shorting bonds, perhaps? Is that the best you can do?
(Crossposted from Twitter)
(That should be “if even one good”, right?)
What do you think of the idea that “consumption by a human” could be considered a task? People may value products / services being consumed by humans because it confers status (e.g. having your artwork be well-received), or because they want humans to have good experiences (aka altruism), or for other reasons.
As long as anyone has a reason to value human consumption of goods / services, it seems like that could play the role of task-that-can’t-be-automated-away.