Claim 1. if >50% of people are not employable anymore and technological progress is >2x faster, a huge fleet of AIs will probably be doing a lot of AI research really well so the pace of conceptual work on AI algorithms is like >100x faster
This is very much unobvious to me, but now that you say this, I realize that I anchored too hard on a specific scenario where the world has gone very hard on just automating away all the economic tasks/roles that can be automated away with advanced robotics and LLMs+++, while humans largely coordinated this fleet in cases that they wouldn’t handle.
But generally, like, to grant the assumption, suppose that 60% are not employable and 40% are employable. Why is this 40% employable? (I think I also took this to be a somewhat stable situation, for some time, not a mean value theorem sort of thing.) Presumably, because there are things that AI still doesn’t do well. Maybe it’s “just” because robotics is annoyingly hard, but it sounds more plausible to me that (also) AI still is not human-thinking-complete, which makes me somewhat sceptical about this massive conceptual algorithm progress speedup.
Unless humans are strictly needed to orchestrate the AIs, but a world where they have thinking coherent enough to make massive algorithmic progress, but incoherent enough to pursue this competently, seems super weird, but, hey, maybe Moravec will come to bite us again!
Claim 2. at most 10 years of human algo thinking followed by a retraining run that takes at most a month would be sufficient to go from top-human-level AI to wildly superhuman AI
You mean something like serial, uninterrupted, focused thinking, like a WBE of a very-high-g AI researcher that doesn’t need sleep?
This is very much unobvious to me, but now that you say this, I realize that I anchored too hard on a specific scenario where the world has gone very hard on just automating away all the economic tasks/roles that can be automated away with advanced robotics and LLMs+++, while humans largely coordinated this fleet in cases that they wouldn’t handle.
But generally, like, to grant the assumption, suppose that 60% are not employable and 40% are employable. Why is this 40% employable? (I think I also took this to be a somewhat stable situation, for some time, not a mean value theorem sort of thing.) Presumably, because there are things that AI still doesn’t do well. Maybe it’s “just” because robotics is annoyingly hard, but it sounds more plausible to me that (also) AI still is not human-thinking-complete, which makes me somewhat sceptical about this massive conceptual algorithm progress speedup.
this makes me want to ask: are you tracking the difference between the event “50 of current human jobs are basically automated” and the event “50 of humans are such that it basically does not make sense to employ them”. like, the former has probably happened multiple times in history, whereas the latter is unprecedented. what you’re saying makes more sense to me if you have the former in mind, but we’re talking about the latter (“people being permanently unemployable”). i have significant probability that you are tracking this correctly already but wanted to check just in case
(I think I also took this to be a somewhat stable situation, for some time, not a mean value theorem sort of thing.)
(to make sure we’re on the same page: in my view, this is unlikely to be a somewhat stable situation)
this makes me want to ask: are you tracking the difference between the event “50 of current human jobs are basically automated” and the event “50 of humans are such that it basically does not make sense to employ them”. like, the former has probably happened multiple times in history, whereas the latter is unprecedented. what you’re saying makes more sense to me if you have the former in mind, but we’re talking about the latter (“people being permanently unemployable”)
Yes, I am talking about 50% people being permanently unemployable, i.e., not being capable of doing any labor that someone would pay meaningful amounts for.
It seems to me that the crux between us is something like: I find a very jagged capability, “Moravec-ian” world plausible, i.e., AI can do lots/most of economically valuable stuff competently, with the amount of human oversight small enough to make 50% of humanity permanently unemployable, while still not being “human-level” on all axes and this remaining stable for a few years at least (which also touches on your claim 2, i.e., an AI that could do all this doesn’t yet exist).
But maybe I’m wrong, and you actually need to be way closer to “human-complete” to do all the boring economic tasks, and AI that is not near-human-complete would not be massively deployable to do them with minimal oversight.
I am now more uncertain, so I will somewhat revise my top-level comment.
This is very much unobvious to me, but now that you say this, I realize that I anchored too hard on a specific scenario where the world has gone very hard on just automating away all the economic tasks/roles that can be automated away with advanced robotics and LLMs+++, while humans largely coordinated this fleet in cases that they wouldn’t handle.
But generally, like, to grant the assumption, suppose that 60% are not employable and 40% are employable. Why is this 40% employable? (I think I also took this to be a somewhat stable situation, for some time, not a mean value theorem sort of thing.) Presumably, because there are things that AI still doesn’t do well. Maybe it’s “just” because robotics is annoyingly hard, but it sounds more plausible to me that (also) AI still is not human-thinking-complete, which makes me somewhat sceptical about this massive conceptual algorithm progress speedup.
Unless humans are strictly needed to orchestrate the AIs, but a world where they have thinking coherent enough to make massive algorithmic progress, but incoherent enough to pursue this competently, seems super weird, but, hey, maybe Moravec will come to bite us again!
You mean something like serial, uninterrupted, focused thinking, like a WBE of a very-high-g AI researcher that doesn’t need sleep?
this makes me want to ask: are you tracking the difference between the event “50
(to make sure we’re on the same page: in my view, this is unlikely to be a somewhat stable situation)
Yes, I am talking about 50% people being permanently unemployable, i.e., not being capable of doing any labor that someone would pay meaningful amounts for.
It seems to me that the crux between us is something like: I find a very jagged capability, “Moravec-ian” world plausible, i.e., AI can do lots/most of economically valuable stuff competently, with the amount of human oversight small enough to make 50% of humanity permanently unemployable, while still not being “human-level” on all axes and this remaining stable for a few years at least (which also touches on your claim 2, i.e., an AI that could do all this doesn’t yet exist).
But maybe I’m wrong, and you actually need to be way closer to “human-complete” to do all the boring economic tasks, and AI that is not near-human-complete would not be massively deployable to do them with minimal oversight.
I am now more uncertain, so I will somewhat revise my top-level comment.
i just meant the usual human AI algo research community doing this for 10 years