Thanks for that, this is helpful. Yes, same genre for sure. According to Eliezer’s response to Holden, tool AI is a synonym of “non-self-improving oracle”. Anyway, whatever we call it, my understanding of the case against tool AI is that (1) we don’t know how to make a safe tool AI (part of Eliezer’s response), and (2) even if we could, it wouldn’t be competitive (Gwern’s response).
I’m trying to contribute to this conversation by giving an intuitive argument for how I’m thinking that both these objections can be overcome, and I’m also trying to be more specific about how the tool AI might be built and how it might work.
More specifically, most (though not 100%) of the reasons that Gwern said tool AI would be uncompetitive are in the category of “self-improving systems are more powerful”. So that’s why I specifically mentioned that a tool AI can be self-improving … albeit indirectly and with a human in the loop.
Thanks for that, this is helpful. Yes, same genre for sure. According to Eliezer’s response to Holden, tool AI is a synonym of “non-self-improving oracle”. Anyway, whatever we call it, my understanding of the case against tool AI is that (1) we don’t know how to make a safe tool AI (part of Eliezer’s response), and (2) even if we could, it wouldn’t be competitive (Gwern’s response).
I’m trying to contribute to this conversation by giving an intuitive argument for how I’m thinking that both these objections can be overcome, and I’m also trying to be more specific about how the tool AI might be built and how it might work.
More specifically, most (though not 100%) of the reasons that Gwern said tool AI would be uncompetitive are in the category of “self-improving systems are more powerful”. So that’s why I specifically mentioned that a tool AI can be self-improving … albeit indirectly and with a human in the loop.