The most astonishing thing to me is what the paper gives as the responses to question 3, part B
“Assume for the purpose of this question that such HLMI will at some point exist. How likely do you then think it is that within (2 years / 30 years) thereafter there will be machine intelligence that greatly surpasses the performance of every human in most professions?”
EETN group, 30 years, median: 55%
What? They think that given a HLMI and 30 years, we have only a 55% chance make a SHLMI? Especially since they (on median) think it’ll take 36 years to get 50% chance of HLMI in the first place (question 2). Do they really think that getting from human to superhuman is such a big step? The Top 100 had a somewhat lower rate (50% chance), but they at least thought it would take 56 years to get HLMI in the first place.
Maybe they’re expecting that the only way the HLMI can be as effective as humans at most professions is by making up with massive efficiency where it does well for the messes it makes in parts it does poorly. But that doesn’t fit with the broad generality of this machine. It can’t have just a few things it does really well.
I think you’re misreading the question. It’s not about going from human to superhuman intelligence, it’s about exceeding human performance in various professions once human-equivalent intelligence is achieved. A lot of professions involve more than just intelligence. Almost all professions involve a large amount of human interaction, as well as the ability to navigate spaces and situations that are designed for humans.
I am highly aware of the exact wording of the question, but thank you for checking. Your point makes it even more unbelieveable.
Things do depend on which professions you see it as handling adequately in the starting case, keeping in mind that it has to be adequate at ‘most’. Also, what you consider a ‘profession’. Lawyer, doctor, scientist, engineer, programmer… manager, carpenter, plumber, mason, police, fire fighter… receptionist, trucker, waiter, cook, retail clerk...
By the time you can be adequate at ‘most’ of those jobs, you’ve got the ‘navigating human spaces’ and ‘interacting with humans’ bit pretty well licked, so the difficulty of these matters is already handled by the time you start the clock.
If on the other hand you cut it off after the second ellipsis (the last few are jobs but not ‘professions’) then you might be able to be only halfway decent at navigating spaces heavily optimized for humans, but you have to already be positively ace at interacting with people.
If on the other hand you cut it off after the first ellipsis, and you’re really just talking hard-core Professions with a capital P, then maaybe it can get away with being effectively immobile to begin with… but even in this extreme case, I can’t see giving it the ability to walk, run, crawl, climb, shimmy, skoonch, and brachiate, or full equivalents, being a task that would span more than 30 years even taking the present as a starting point—and we haven’t got HLMI today.
The most astonishing thing to me is what the paper gives as the responses to question 3, part B
“Assume for the purpose of this question that such HLMI will at some point exist. How likely do you then think it is that within (2 years / 30 years) thereafter there will be machine intelligence that greatly surpasses the performance of every human in most professions?”
EETN group, 30 years, median: 55%
What? They think that given a HLMI and 30 years, we have only a 55% chance make a SHLMI? Especially since they (on median) think it’ll take 36 years to get 50% chance of HLMI in the first place (question 2). Do they really think that getting from human to superhuman is such a big step? The Top 100 had a somewhat lower rate (50% chance), but they at least thought it would take 56 years to get HLMI in the first place.
Maybe they’re expecting that the only way the HLMI can be as effective as humans at most professions is by making up with massive efficiency where it does well for the messes it makes in parts it does poorly. But that doesn’t fit with the broad generality of this machine. It can’t have just a few things it does really well.
I feel like even experts reject anything “superhuman” based on stuff like absurdity heuristic or not thinking intelligence is a thing etc.
And now you’ve learned how little we can trust expert consensus on these issues.
I think you’re misreading the question. It’s not about going from human to superhuman intelligence, it’s about exceeding human performance in various professions once human-equivalent intelligence is achieved. A lot of professions involve more than just intelligence. Almost all professions involve a large amount of human interaction, as well as the ability to navigate spaces and situations that are designed for humans.
I am highly aware of the exact wording of the question, but thank you for checking. Your point makes it even more unbelieveable.
Things do depend on which professions you see it as handling adequately in the starting case, keeping in mind that it has to be adequate at ‘most’. Also, what you consider a ‘profession’. Lawyer, doctor, scientist, engineer, programmer… manager, carpenter, plumber, mason, police, fire fighter… receptionist, trucker, waiter, cook, retail clerk...
By the time you can be adequate at ‘most’ of those jobs, you’ve got the ‘navigating human spaces’ and ‘interacting with humans’ bit pretty well licked, so the difficulty of these matters is already handled by the time you start the clock.
If on the other hand you cut it off after the second ellipsis (the last few are jobs but not ‘professions’) then you might be able to be only halfway decent at navigating spaces heavily optimized for humans, but you have to already be positively ace at interacting with people.
If on the other hand you cut it off after the first ellipsis, and you’re really just talking hard-core Professions with a capital P, then maaybe it can get away with being effectively immobile to begin with… but even in this extreme case, I can’t see giving it the ability to walk, run, crawl, climb, shimmy, skoonch, and brachiate, or full equivalents, being a task that would span more than 30 years even taking the present as a starting point—and we haven’t got HLMI today.