I think it’s true that Séb doesn’t believe in AGI/ASI in the sense that I mean: see his fourth response here. Also see e.g. this model he posted which argues that comparative advantage implies postASI humans will still have jobs (which doesn’t make sense if you think ASI can use the inputs that humans require much more efficiently than the humans do).
Seems very reasonable to be skeptical of many of AI 2040s proposals despite believing in AGI/ASI. I’d be curious to hear more (though perhaps you should post those on the main post, not here, if you feel interested :) ).
Re clarifing the quotes; reading them now I still think they are all misleading but to varying degrees and probably won’t spend more time further getting into it because there’s a lot on my plate right now, but if others think it’d be useful for me or someone else at AIFP to do another pass here trying to further clarify let us know (e.g. by reacting or commenting here).
I don’t think belief in ASI necessarily implies belief in all of:
> (a) extremely fast diffusion and societal transformation, (b) a view that all profits accrue maximally to the labs, (c) that the prescriptions advanced in the essay (like expropriations and forced IP diffusion) have minimal impacts on said profits; (d) the claim that you get explosive GDP growth very soon, (e) that ‘de facto’ nationalisation and profit redistribution through UBI is an optimal response.
Or did you mean some other part of his fourth response?
I also don’t think it requires you believe that no comparative advantage for humans post-ASI exists: positional goods are still a thing, a price premium for human-produced goods is very plausible (and positing that it won’t happen is not a question of capabilities).
I think it’s a common move to claim that people with a different conception of a post-ASI trajectory “don’t believe in” ASI, but when digging into it usually they don’t disagree on raw capabilities, just on what those raw capabilities imply the ASI would be able to do in the world.
(a) extremely fast diffusion and societal transformation, (b) a view that all profits accrue maximally to the labs, (c) that the prescriptions advanced in the essay (like expropriations and forced IP diffusion) have minimal impacts on said profits; (d) the claim that you get explosive GDP growth very soon, (e) that ‘de facto’ nationalisation and profit redistribution through UBI is an optimal response.
As I said in the post, this part is largely a strawman of our views. The part I mostly meant was “Nor do I think we will get real GDP growth of 50% in 2032” and “the model moves far too quickly from AIs being able to perform tasks to robots being reliable, legally deployable, organisationally integrated substitutes for almost all labour, and from there to a closed-loop reproduction of capital”. Perhaps what he means is that there will be delays, but eventually we’ll get 50% GDP growth?
I think it’s a common move to claim that people with a different conception of a post-ASI trajectory “don’t believe in” ASI, but when digging into it usually they don’t disagree on raw capabilities, just on what those raw capabilities imply the ASI would be able to do in the world.
I tend to find the exact opposite is true. People love claiming that the real disagreement is real world bottlenecks, but usually its really differences in capability expectations. I think the AIs will be wildly superintelligent, with vast quantities operating at the equivalent of 100x or 1000x human speeds, also with a much higher qualitative intelligence due to things like knowing way more than any human can know, having much more experience than any human can every get, and having a physically much larger and more connected brain such that they are able to make discoveries and model things that are far too complicated for humans.
This is obviously an extreme milestone; we outline earlier milestones here: https://ai-rates-calculator.vercel.app/, for example. I would encourage people to make forecasts of when they think these specific milestones will be crossed (which might be “in hundreds of years or never”). If earlier than that, maybe its just a timelines / takeoff speed disagreement.
I kind of expect that many people would argue that this is the wrong ontology for thinking about AI capability progressions, and that actually, it’s somehow going to be more diffuse/multipolar or something, in a way where thinking in these terms isn’t useful for modeling the world, in which case I’d love to hear a better frame.
Still, I think that most people with very different views would tend to disagree that the notion of ” superintelligent” AI that I outlined above would happen but will have small effects on the world. I think this view is extremely hard to make coherent. Once we’ve got AIs like that, the cognitive labour supply would become enourmous, such that almost all the cognitive labour happening would be AIs, not humans.
Totally appreciate you guys have a lot on your plate right now! At least two of the examples I mentioned in my top-level comment didn’t seem like misrepresentations to me (though two of them did), which makes me confused as to how much of this post is accurate if it really is saying that all of the quoted examples are misrepresentations (hence the reason for my comment requesting that clarification). I haven’t carefully dug into every claim myself but overall my sense is that this post very heavily blurs the line between vehement disagreement and actual misrepresentation such that I don’t feel there’s a clear takeaway for me about how much of each is going on in Séb’s critique.
I think it’s true that Séb doesn’t believe in AGI/ASI in the sense that I mean: see his fourth response here. Also see e.g. this model he posted which argues that comparative advantage implies postASI humans will still have jobs (which doesn’t make sense if you think ASI can use the inputs that humans require much more efficiently than the humans do).
Seems very reasonable to be skeptical of many of AI 2040s proposals despite believing in AGI/ASI. I’d be curious to hear more (though perhaps you should post those on the main post, not here, if you feel interested :) ).
Re clarifing the quotes; reading them now I still think they are all misleading but to varying degrees and probably won’t spend more time further getting into it because there’s a lot on my plate right now, but if others think it’d be useful for me or someone else at AIFP to do another pass here trying to further clarify let us know (e.g. by reacting or commenting here).
I don’t think belief in ASI necessarily implies belief in all of:
> (a) extremely fast diffusion and societal transformation, (b) a view that all profits accrue maximally to the labs, (c) that the prescriptions advanced in the essay (like expropriations and forced IP diffusion) have minimal impacts on said profits; (d) the claim that you get explosive GDP growth very soon, (e) that ‘de facto’ nationalisation and profit redistribution through UBI is an optimal response.
Or did you mean some other part of his fourth response?
I also don’t think it requires you believe that no comparative advantage for humans post-ASI exists: positional goods are still a thing, a price premium for human-produced goods is very plausible (and positing that it won’t happen is not a question of capabilities).
I think it’s a common move to claim that people with a different conception of a post-ASI trajectory “don’t believe in” ASI, but when digging into it usually they don’t disagree on raw capabilities, just on what those raw capabilities imply the ASI would be able to do in the world.
As I said in the post, this part is largely a strawman of our views. The part I mostly meant was “Nor do I think we will get real GDP growth of 50% in 2032” and “the model moves far too quickly from AIs being able to perform tasks to robots being reliable, legally deployable, organisationally integrated substitutes for almost all labour, and from there to a closed-loop reproduction of capital”. Perhaps what he means is that there will be delays, but eventually we’ll get 50% GDP growth?
I tend to find the exact opposite is true. People love claiming that the real disagreement is real world bottlenecks, but usually its really differences in capability expectations. I think the AIs will be wildly superintelligent, with vast quantities operating at the equivalent of 100x or 1000x human speeds, also with a much higher qualitative intelligence due to things like knowing way more than any human can know, having much more experience than any human can every get, and having a physically much larger and more connected brain such that they are able to make discoveries and model things that are far too complicated for humans.
This is obviously an extreme milestone; we outline earlier milestones here: https://ai-rates-calculator.vercel.app/, for example. I would encourage people to make forecasts of when they think these specific milestones will be crossed (which might be “in hundreds of years or never”). If earlier than that, maybe its just a timelines / takeoff speed disagreement.
I kind of expect that many people would argue that this is the wrong ontology for thinking about AI capability progressions, and that actually, it’s somehow going to be more diffuse/multipolar or something, in a way where thinking in these terms isn’t useful for modeling the world, in which case I’d love to hear a better frame.
Still, I think that most people with very different views would tend to disagree that the notion of ” superintelligent” AI that I outlined above would happen but will have small effects on the world. I think this view is extremely hard to make coherent. Once we’ve got AIs like that, the cognitive labour supply would become enourmous, such that almost all the cognitive labour happening would be AIs, not humans.
Totally appreciate you guys have a lot on your plate right now! At least two of the examples I mentioned in my top-level comment didn’t seem like misrepresentations to me (though two of them did), which makes me confused as to how much of this post is accurate if it really is saying that all of the quoted examples are misrepresentations (hence the reason for my comment requesting that clarification). I haven’t carefully dug into every claim myself but overall my sense is that this post very heavily blurs the line between vehement disagreement and actual misrepresentation such that I don’t feel there’s a clear takeaway for me about how much of each is going on in Séb’s critique.