I’m definitely not trying to attack anyone (and you’re right my comment could be read that way). But I’m also not just curious. I figured this was the answer. Lots of time spent thinking, writing and producing influential publications on FAI is about all the qualifications one can reasonably expect (producing a provable mathematical formalization of friendliness is the kind of thing no one is qualified to do before they do it and the AI field in general is relatively new and small). And Eliezer is obviously a really smart guy. He’s probably even the most likely person to solve it. But the effort to address the friendliness issue seems way too focused on him and the people around him. You shouldn’t expect any one person to solve a Hard problem. Insight isn’t that predictable especially when no one in the field has solved comparable problems before. Maybe Einstein was the best bet to formulate a unified field theory but a) he never did and b) he had actually had comparable insights in the past. Part of the focus on Eliezer is just an institutional and financial thing, but he and a lot of people here seem to encourage this state of affairs.
No one looks at open problems in other fields this way.
No one looks at open problems in other fields this way.
Yes, the situation isn’t normal or good. But this isn’t a balanced comparison, since we don’t currently have a field, too few people understand the problem and had seriously thought about it. This gradually changes, and I expect will be visibly less of a problem in another 10 years.
I may have an incorrect impression, but SIAI or at least Eliezer’s department seems to have a self-image comparable to the Manhattan project rather than early pioneers of a scientific field.
I may have an incorrect impression, but SIAI or at least Eliezer’s department seems to have a self-image comparable to the Manhattan project
Eliezer has said: “I have a policy of keeping my thoughts on Friendly AI to the object level, and not worrying about how important or unimportant that makes me.” Your call as to whether you believe that. (The rest of that post, and some of his other posts in that discussion, address some points similar to those that you raised.)
That said, “self-image comparable to the Manhattan project” is an unusually generous ascription of humility to SIAI and Eliezer. :P
...producing a provable mathematical formalization of friendliness [...] And Eliezer is obviously a really smart guy. He’s probably even the most likely person to solve it.
I haven’t seen any proves of his math skills that would justify this statement. By what evidence have you arrived at the conclusion that he can do it at all, even approach it? The sequences and the SIAI publications certainly show that he was able to compile a bunch of existing ideas into a coherent framework of rationality, yet there is not much novelty to be found anywhere.
Which statement are you talking about? Saying someone is the most likely person to do something is not the same as saying they are likely to do it. You haven’t said anything in this comment than I disagree with so I don’t understand what we’re disputing.
I’m definitely not trying to attack anyone (and you’re right my comment could be read that way). But I’m also not just curious. I figured this was the answer. Lots of time spent thinking, writing and producing influential publications on FAI is about all the qualifications one can reasonably expect (producing a provable mathematical formalization of friendliness is the kind of thing no one is qualified to do before they do it and the AI field in general is relatively new and small). And Eliezer is obviously a really smart guy. He’s probably even the most likely person to solve it. But the effort to address the friendliness issue seems way too focused on him and the people around him. You shouldn’t expect any one person to solve a Hard problem. Insight isn’t that predictable especially when no one in the field has solved comparable problems before. Maybe Einstein was the best bet to formulate a unified field theory but a) he never did and b) he had actually had comparable insights in the past. Part of the focus on Eliezer is just an institutional and financial thing, but he and a lot of people here seem to encourage this state of affairs.
No one looks at open problems in other fields this way.
Yes, the situation isn’t normal or good. But this isn’t a balanced comparison, since we don’t currently have a field, too few people understand the problem and had seriously thought about it. This gradually changes, and I expect will be visibly less of a problem in another 10 years.
I may have an incorrect impression, but SIAI or at least Eliezer’s department seems to have a self-image comparable to the Manhattan project rather than early pioneers of a scientific field.
Eliezer’s past remarks seem to have pointed to a self-image comparable to the Manhatten project. However, according the new SIAI Overview:
They want to become comparable to the Manhattan project, in part by recruiting additional FAI researchers. They do not claim to be at that stage now.
Eliezer has said: “I have a policy of keeping my thoughts on Friendly AI to the object level, and not worrying about how important or unimportant that makes me.” Your call as to whether you believe that. (The rest of that post, and some of his other posts in that discussion, address some points similar to those that you raised.)
That said, “self-image comparable to the Manhattan project” is an unusually generous ascription of humility to SIAI and Eliezer. :P
I haven’t seen any proves of his math skills that would justify this statement. By what evidence have you arrived at the conclusion that he can do it at all, even approach it? The sequences and the SIAI publications certainly show that he was able to compile a bunch of existing ideas into a coherent framework of rationality, yet there is not much novelty to be found anywhere.
Which statement are you talking about? Saying someone is the most likely person to do something is not the same as saying they are likely to do it. You haven’t said anything in this comment than I disagree with so I don’t understand what we’re disputing.
Great comment.