tldr: I think it matters differently when something is discontinuitous with linear vs exponential vs hyperbolic curves.
Some thoughts copied over from an earlier conversation about this.
Reading this and some of Paul’s related thoughts, I experienced some confusion about why we care about discontinuity in the first place. Here are my thoughts on that.
Caring about discontinuity is similar but not identical to “why do we care about takeoff speed?”. ESRogs mentions in this comment in the related Paul thread that we might care about:
How much of a lead will one team need to have over others in order to have a decisive strategic advantage...
in wall clock time? in economic doublings time?
How much time do we have to solve hard alignment problems...
in wall clock time? in economic doublings time?
A related issue is how surprising an advance will be, in a fashion that disrupts our ability to manage it.
A problem the AI safety crowd has faced is that people don’t intuitively understand exponentials. In most of our lives, things move fairly linearly. The “one grain of rice, doubled per day” parable works because the King has no conception of how bad things are going to get and how fast.
So if there’s a discontinuity in a linear curve, the average human will be surprised, but only once, and then the curve goes back to it’s usual progress and things are mostly okay.
If a curve that looked linear-ish turns out to be exponential, that human may find themselves surprised if they’re not paying attention, once the curve moves past an inflection point. And more surprised if the curve was hyperbolic. (If you are paying attention I’m not sure whether it matters whether the curve is exponential or hyperbolic)
If an exponential curve has a discontinuity, it’s even more surprising. The average human is hit both with a curve arcing upward faster than they expected, and with that curve suddenly jumping upward.
Now, I’m not sure how relevant this is to Leaders of Industry in the AI world – presumably at least some are paying attention. I’m not actually sure how much attention I expect them to be paying (there’s a lot of things to pay attention to, and if we’re at the early stage of a hyperbolic curve it’s a plausible mistake to assume it’s linear and/or “only” exponential).
(I think it matters a lot more whether industry leaders are paying attention than whether Joe/Jane Public is. It seems like a major reason discontinuity matters (or whether DeepMind et all think it matters, or think other organizations think it matters) has to do with both arms races, and whether or not it’s plausible that we end up in a multipolar scenario vs a singleton)
Relatedly, the thing that actually prompted the above was looking at this graph, which Katja and Sarah Constantin have both referenced:
And not being sure how to think about Chess ratings and whether this discontinuity is better thought of as linear or exponential or what. Or, put another way: insofar as being good at Chess has anything to do with fully general artificial intelligence, is that jump mean we’re a little closer or a lot closer?)
At this point I think it’s clear being good at Chess is fairly distinct from being good at fully general intelligence so that particular framing doesn’t matter, but one might have expected it to matter before much progress on Chess was made.
This brought to mind a general confusion I’ve had, which is that I see people taking data of various sorts of curves, and plotting them on various sorts of graphs, and sometimes from context it’s hard to tell how important a trend is.
And not being sure how to think about Chess ratings and whether this discontinuity is better thought of as linear or exponential or what.
Are you talking about the overall trend, or the discontinuity itself?
It doesn’t seem like it would make sense to talk about the discontinuity itself as linear / exponential, since it’s defined by just two points: the point before the jump, and the point after the jump. You could fit a line through two points, you could fit an exponential through two points, you could fit anything!
(If you had a trend that switched from being linear to being exponential, that would be a different story. But this graph doesn’t look like that to me.)
Are you talking about the overall trend, or the discontinuity itself?
I was mostly talking about the overall trend, although I have additional thoughts on your point about the point-of-discontinuity.
(epistemic status: a bit outside my comfort zone. I feel confident but wouldn’t be too surprised if someone who thinks about this more than me responded in a way that updated me considerably. But, I think I may be communicating a point that has reverse inferential distance – i.e the points I’m making are so obvious that they don’t seem relevant to the discussion, and my point is that if you’re not used to thinking in exponential terms they aren’t obvious, so this subthread may be most useful to people who happen to be feel confused or that things are unintuitive in the way I feel right now)
You could fit a line through two points, you could fit an exponential through two points, you could fit anything!
I mean, presumably there are more data points you could (at least hypothetically) have included, in which it’s not literally a single discontinuity, but a brief switch to a sharp increase in progress, followed by a return to something closer to the original curve. I’m not sure about the technical definition of discontinuity, but in a world where the graph had a point for each month instead of year, but the year of 2007 still had such a sharp uptick, the point doesn’t stop being interesting.
Since the Chess graph is uniquely confusing (hence my original confusion), I’d answer the rest of your question with, say, a more generic economic growth model.
If the economy were growing linearly, and then had a brief spike, and then returned to growing linearly at roughly the same rate, that’s one kind of interesting.
The fact that the economy grows exponential is a different kind of interesting, which layfolk routinely make bad choices due to poor intuitions about. (i.e. this is why investing is a much better idea that it seems, and why making tradeoffs that involve half-percent sacrifices to economic growth are a big deal. If you’re used to thinking about it this way it may not longer seem interesting, but, like, there are whole courses explaining this concept because it’s non-obvious)
If the economy is growing exponentially, and there’s a discontinuity where for one year it grows much more rapidly, that’s a third kind of interesting, and it’s in turn different interesting whether growth slows back down such that it seems like it’s at a similar rate to what we had before the spike, or continues as the spike had basically let you skip several years and then continue at an even faster rate.
tldr: I think it matters differently when something is discontinuitous with linear vs exponential vs hyperbolic curves.
Some thoughts copied over from an earlier conversation about this.
Reading this and some of Paul’s related thoughts, I experienced some confusion about why we care about discontinuity in the first place. Here are my thoughts on that.
Caring about discontinuity is similar but not identical to “why do we care about takeoff speed?”. ESRogs mentions in this comment in the related Paul thread that we might care about:
A related issue is how surprising an advance will be, in a fashion that disrupts our ability to manage it.
A problem the AI safety crowd has faced is that people don’t intuitively understand exponentials. In most of our lives, things move fairly linearly. The “one grain of rice, doubled per day” parable works because the King has no conception of how bad things are going to get and how fast.
So if there’s a discontinuity in a linear curve, the average human will be surprised, but only once, and then the curve goes back to it’s usual progress and things are mostly okay.
If a curve that looked linear-ish turns out to be exponential, that human may find themselves surprised if they’re not paying attention, once the curve moves past an inflection point. And more surprised if the curve was hyperbolic. (If you are paying attention I’m not sure whether it matters whether the curve is exponential or hyperbolic)
If an exponential curve has a discontinuity, it’s even more surprising. The average human is hit both with a curve arcing upward faster than they expected, and with that curve suddenly jumping upward.
Now, I’m not sure how relevant this is to Leaders of Industry in the AI world – presumably at least some are paying attention. I’m not actually sure how much attention I expect them to be paying (there’s a lot of things to pay attention to, and if we’re at the early stage of a hyperbolic curve it’s a plausible mistake to assume it’s linear and/or “only” exponential).
(I think it matters a lot more whether industry leaders are paying attention than whether Joe/Jane Public is. It seems like a major reason discontinuity matters (or whether DeepMind et all think it matters, or think other organizations think it matters) has to do with both arms races, and whether or not it’s plausible that we end up in a multipolar scenario vs a singleton)
Relatedly, the thing that actually prompted the above was looking at this graph, which Katja and Sarah Constantin have both referenced:
And not being sure how to think about Chess ratings and whether this discontinuity is better thought of as linear or exponential or what. Or, put another way: insofar as being good at Chess has anything to do with fully general artificial intelligence, is that jump mean we’re a little closer or a lot closer?)
At this point I think it’s clear being good at Chess is fairly distinct from being good at fully general intelligence so that particular framing doesn’t matter, but one might have expected it to matter before much progress on Chess was made.
This brought to mind a general confusion I’ve had, which is that I see people taking data of various sorts of curves, and plotting them on various sorts of graphs, and sometimes from context it’s hard to tell how important a trend is.
Are you talking about the overall trend, or the discontinuity itself?
It doesn’t seem like it would make sense to talk about the discontinuity itself as linear / exponential, since it’s defined by just two points: the point before the jump, and the point after the jump. You could fit a line through two points, you could fit an exponential through two points, you could fit anything!
(If you had a trend that switched from being linear to being exponential, that would be a different story. But this graph doesn’t look like that to me.)
Have I misunderstood what you’re saying?
I was mostly talking about the overall trend, although I have additional thoughts on your point about the point-of-discontinuity.
(epistemic status: a bit outside my comfort zone. I feel confident but wouldn’t be too surprised if someone who thinks about this more than me responded in a way that updated me considerably. But, I think I may be communicating a point that has reverse inferential distance – i.e the points I’m making are so obvious that they don’t seem relevant to the discussion, and my point is that if you’re not used to thinking in exponential terms they aren’t obvious, so this subthread may be most useful to people who happen to be feel confused or that things are unintuitive in the way I feel right now)
I mean, presumably there are more data points you could (at least hypothetically) have included, in which it’s not literally a single discontinuity, but a brief switch to a sharp increase in progress, followed by a return to something closer to the original curve. I’m not sure about the technical definition of discontinuity, but in a world where the graph had a point for each month instead of year, but the year of 2007 still had such a sharp uptick, the point doesn’t stop being interesting.
Since the Chess graph is uniquely confusing (hence my original confusion), I’d answer the rest of your question with, say, a more generic economic growth model.
If the economy were growing linearly, and then had a brief spike, and then returned to growing linearly at roughly the same rate, that’s one kind of interesting.
The fact that the economy grows exponential is a different kind of interesting, which layfolk routinely make bad choices due to poor intuitions about. (i.e. this is why investing is a much better idea that it seems, and why making tradeoffs that involve half-percent sacrifices to economic growth are a big deal. If you’re used to thinking about it this way it may not longer seem interesting, but, like, there are whole courses explaining this concept because it’s non-obvious)
If the economy is growing exponentially, and there’s a discontinuity where for one year it grows much more rapidly, that’s a third kind of interesting, and it’s in turn different interesting whether growth slows back down such that it seems like it’s at a similar rate to what we had before the spike, or continues as the spike had basically let you skip several years and then continue at an even faster rate.