Thanks for writing this and posting it here. I for one am a big fan of the “Bostromian position” as you call it, and moreover I think Stuart Russell is too. (“Human Compatible” is making basically the same points as “Superintelligence,” only in a dumbed-down and streamlined manner, with lots of present-day examples to illustrate.) So I don’t think your dismissal of positions 1 and 2 is fair. But I’m glad to see dialogue happening between the likes of me and the likes of you.
Moreover, I think you are actually right about Scott Alexander’s GPT-2 chess thingy. As you’ve explained, we knew neural nets could do this sort of thing already, and so we shouldn’t be too surprised if GPT-2 can do it too with a little retraining.
I suppose, in Scott’s defense, perhaps he wasn’t surprised but rather just interested, and using the fact that GPT-2 can play chess to argue for some further claim about how useful neural nets are in general and how soon AGI will appear. But I currently prefer your take.
“Human Compatible” is making basically the same points as “Superintelligence,” only in a dumbed-down and streamlined manner, with lots of present-day examples to illustrate.
I do not agree with this. I think the arguments in Human Compatible are more convincing than the ones in Superintelligence (mostly because they make fewer questionable assumptions).
(I agree that Stuart probably does agree somewhat with the “Bostromian position”.)
Interesting. Well, I imagine you don’t have the time right now, but I just want to register that I’d love to hear more about this. What questionable assumptions does Superintelligence make, that aren’t made by Human Compatible? (This request for info goes out to everyone, not just Rohin)
Yes, I guess I should have made the clarification about that, I don’t think Stuart Russell is necessarily much divergent from Bostrom in his views. Rather, he’s most poniente arguments seem not to be very related to that view, so I think his books is a good guide for what I labeled as the second view in the article.
But he certainly tries to uphold both.
However the article was already too long and going into that would have made it even longer.… in hindsight I’ve decided to just split it into two, but the version here I shall leave as is.
Thanks for writing this and posting it here. I for one am a big fan of the “Bostromian position” as you call it, and moreover I think Stuart Russell is too. (“Human Compatible” is making basically the same points as “Superintelligence,” only in a dumbed-down and streamlined manner, with lots of present-day examples to illustrate.) So I don’t think your dismissal of positions 1 and 2 is fair. But I’m glad to see dialogue happening between the likes of me and the likes of you.
Moreover, I think you are actually right about Scott Alexander’s GPT-2 chess thingy. As you’ve explained, we knew neural nets could do this sort of thing already, and so we shouldn’t be too surprised if GPT-2 can do it too with a little retraining.
I suppose, in Scott’s defense, perhaps he wasn’t surprised but rather just interested, and using the fact that GPT-2 can play chess to argue for some further claim about how useful neural nets are in general and how soon AGI will appear. But I currently prefer your take.
I do not agree with this. I think the arguments in Human Compatible are more convincing than the ones in Superintelligence (mostly because they make fewer questionable assumptions).
(I agree that Stuart probably does agree somewhat with the “Bostromian position”.)
Interesting. Well, I imagine you don’t have the time right now, but I just want to register that I’d love to hear more about this. What questionable assumptions does Superintelligence make, that aren’t made by Human Compatible? (This request for info goes out to everyone, not just Rohin)
Yes, I guess I should have made the clarification about that, I don’t think Stuart Russell is necessarily much divergent from Bostrom in his views. Rather, he’s most poniente arguments seem not to be very related to that view, so I think his books is a good guide for what I labeled as the second view in the article.
But he certainly tries to uphold both.
However the article was already too long and going into that would have made it even longer.… in hindsight I’ve decided to just split it into two, but the version here I shall leave as is.