Eliezer: One comment is that I don’t particularly trust your capability to assess the insights or mental capabilities of people who think very differently from yourself. It may be that the people whose intelligence you most value (who you rate as residing on “high levels”, to quasi-borrow your terminology) are those who are extremely talented at the kind of thinking you personally most value. Yet, there may be many different sorts of intelligent human thinking, some of which you may not excel at, may understand relatively little of, and may not be particularly good at assessing in others. And, it’s not yet clear whether the style of intelligence that you favor (or the slightly different one that I tend to intuitively, and by personality-bias, favor) is the one that is most likely to lead to powerful, beneficial AGI … or whether some other style of intelligence may be more effective in this regard....
I note again that objective definitions of general intelligence don’t really exist except in the limit of massive computational processing power (and even there, they’re controversial). So, assessing intelligence or capability in practice is a subtle matter … and I don’t particularly trust your analysis of intelligence in terms of a hierarchy of levels. I guess human intelligence is more mess, heterarchical and multifaceted than that. Of course, you can meaningfully construct hierarchies of intelligence in various areas, such as “mathematical theorem proving” or “theorem proving in continuous-variable analysis and related branches of math” … or, say, “biology experimental design” or “software design”, etc. But, when dealing with something like AGI that is poorly understood and may be amenable to a variety of different approaches, it’s hard to say which of these domain-specific intelligences are going to be most critical to the effective solution of the AGI problem.
Maybe one of these scientists whom you dismiss as “mediocre level” according to the particular aspects of intelligence that you value most, are actually “high level” according to other aspects of intelligence that you aren’t able to recognize and evaluate so accurately … and maybe some of these other aspects will turn out to be MORE valuable for the creation of AGI.
I’m not saying I have a strong feeling this is the case … I’m just saying “maybe”....
Compared to you, I think I have a bit more humility about my capability to recognize what another person’s capabilities really are. Yes, I can see how well they do on a test, or how clever they are in a conversation … or what papers they publish. But how do I know what’s in their mind, that is not revealed to me explicitly due to the strictures of their personality or culture? How do I know what is in their statements or works that I’m not well-suited to recognize due to my own particular biases and limitations?
When I have to choose which scientist or engineer to hire or collaborate with, then I just make my best judgments … and if I miss out on someone great due to my own limitations of vision, so be it … but I personally tend to be more hesitant to consider either my own gut-level assessment of another’s abilities, or performance on narrowly-specified test instruments, or success in social rituals like paper-publishing or university, as fundamentally indicative of someone’s general intelligence or intellectual capability...
Eliezer: One comment is that I don’t particularly trust your capability to assess the insights or mental capabilities of people who think very differently from yourself. It may be that the people whose intelligence you most value (who you rate as residing on “high levels”, to quasi-borrow your terminology) are those who are extremely talented at the kind of thinking you personally most value. Yet, there may be many different sorts of intelligent human thinking, some of which you may not excel at, may understand relatively little of, and may not be particularly good at assessing in others. And, it’s not yet clear whether the style of intelligence that you favor (or the slightly different one that I tend to intuitively, and by personality-bias, favor) is the one that is most likely to lead to powerful, beneficial AGI … or whether some other style of intelligence may be more effective in this regard....
I note again that objective definitions of general intelligence don’t really exist except in the limit of massive computational processing power (and even there, they’re controversial). So, assessing intelligence or capability in practice is a subtle matter … and I don’t particularly trust your analysis of intelligence in terms of a hierarchy of levels. I guess human intelligence is more mess, heterarchical and multifaceted than that. Of course, you can meaningfully construct hierarchies of intelligence in various areas, such as “mathematical theorem proving” or “theorem proving in continuous-variable analysis and related branches of math” … or, say, “biology experimental design” or “software design”, etc. But, when dealing with something like AGI that is poorly understood and may be amenable to a variety of different approaches, it’s hard to say which of these domain-specific intelligences are going to be most critical to the effective solution of the AGI problem.
Maybe one of these scientists whom you dismiss as “mediocre level” according to the particular aspects of intelligence that you value most, are actually “high level” according to other aspects of intelligence that you aren’t able to recognize and evaluate so accurately … and maybe some of these other aspects will turn out to be MORE valuable for the creation of AGI.
I’m not saying I have a strong feeling this is the case … I’m just saying “maybe”....
Compared to you, I think I have a bit more humility about my capability to recognize what another person’s capabilities really are. Yes, I can see how well they do on a test, or how clever they are in a conversation … or what papers they publish. But how do I know what’s in their mind, that is not revealed to me explicitly due to the strictures of their personality or culture? How do I know what is in their statements or works that I’m not well-suited to recognize due to my own particular biases and limitations?
When I have to choose which scientist or engineer to hire or collaborate with, then I just make my best judgments … and if I miss out on someone great due to my own limitations of vision, so be it … but I personally tend to be more hesitant to consider either my own gut-level assessment of another’s abilities, or performance on narrowly-specified test instruments, or success in social rituals like paper-publishing or university, as fundamentally indicative of someone’s general intelligence or intellectual capability...
-- Ben G