MIRI leaders including Eliezer Yudkowsky and Nate Soares told me that this was overly naive, that DeepMind would not stop dangerous research even if good reasons for this could be given.
I have no memory of saying this to Jessica; this of itself is not strong evidence because my autobiographical memory is bad, but it also doesn’t sound like something I would say. I generally credit Demis Hassabis as being more clueful than many, though unfortunately not on quite the same page. Adjacent things that could possibly have actually been said in reality might include “It’s not clear that Demis has the power to prevent Google’s CEO from turning up the dial on an AGI even if Demis thinks that’s a bad idea” or “Deepmind has recruited a lot of people who would strongly protest reduced publications, given their career incentives and the impression they had when they signed up” or maybe something something Law of Continued Failure they already have strong reasons not to advance the field so why would providing them with stronger ones help.
Therefore (they said) it was reasonable to develop precursors to AGI in-house to compete with organizations such as DeepMind in terms of developing AGI first.
I haven’t been shy over the course of my entire career about saying that I’d do this if I could; it’s looking less hopeful in 2020 than in 2010 due to the trajectory of machine learning and timelines generally looking shorter.
So I was being told to consider people at other AI organizations to be intractably wrong, people who it makes more sense to compete with than to treat as participants in a discourse.
Not something I’d have said, and the sort of statement which would make a bunch of readers think “Oh Eliezer said that explicitly” but with a nice little motte of “Oh, I just meant that was the implication somebody could have took from other things Eliezer said.”
In case it refreshes your memory, this was in a research retreat, we were in a living room on couches, you and I and Nate were there, Critch and Ryan Carey were probably there, I was saying that convincing DeepMind people to care about alignment was a good plan, people were saying that was overly naive and competition was a better approach. I believe Nate specifically said something about Demis saying that he couldn’t stop DeepMind researchers from publishing dangerous/unaligned AI things even if he tried. Even if Demis can be reasoned with, that doesn’t imply DeepMind as a whole can be reasoned with, since DeepMind also includes and is driven by these researchers who Demis doesn’t think he can reason with.
Sounds like something that could have happened, sure, I wouldn’t be surprised to hear Critch or Carey confirm that version of things. A retreat with non-MIRI people present, and nuanced general discussion on that topic happening, is a very different event to have actually happened than the impression this post leaves in the mind of the reader.
Do you have a take on Shane Legg? Or any insight to his safety efforts? In his old blog and the XiXiDu interview, he was pretty solid on alignment, back when it was far harder to say such things publicly. And he made this comment in this post, just before starting DeepMind:
A better approach would be to act as a parent organisation, a kind of AGI VC company, that backs a number of promising teams. Teams that fail to make progress get dropped and new teams with new ideas are picked up. General ideas of AGI safety are also developed in the background until such a time when one of the teams starts to make serious progress. At this time the focus would be to make the emerging AGI design as safe as possible.
I have no memory of saying this to Jessica; this of itself is not strong evidence because my autobiographical memory is bad, but it also doesn’t sound like something I would say. I generally credit Demis Hassabis as being more clueful than many, though unfortunately not on quite the same page. Adjacent things that could possibly have actually been said in reality might include “It’s not clear that Demis has the power to prevent Google’s CEO from turning up the dial on an AGI even if Demis thinks that’s a bad idea” or “Deepmind has recruited a lot of people who would strongly protest reduced publications, given their career incentives and the impression they had when they signed up” or maybe something something Law of Continued Failure they already have strong reasons not to advance the field so why would providing them with stronger ones help.
I haven’t been shy over the course of my entire career about saying that I’d do this if I could; it’s looking less hopeful in 2020 than in 2010 due to the trajectory of machine learning and timelines generally looking shorter.
Not something I’d have said, and the sort of statement which would make a bunch of readers think “Oh Eliezer said that explicitly” but with a nice little motte of “Oh, I just meant that was the implication somebody could have took from other things Eliezer said.”
In case it refreshes your memory, this was in a research retreat, we were in a living room on couches, you and I and Nate were there, Critch and Ryan Carey were probably there, I was saying that convincing DeepMind people to care about alignment was a good plan, people were saying that was overly naive and competition was a better approach. I believe Nate specifically said something about Demis saying that he couldn’t stop DeepMind researchers from publishing dangerous/unaligned AI things even if he tried. Even if Demis can be reasoned with, that doesn’t imply DeepMind as a whole can be reasoned with, since DeepMind also includes and is driven by these researchers who Demis doesn’t think he can reason with.
Sounds like something that could have happened, sure, I wouldn’t be surprised to hear Critch or Carey confirm that version of things. A retreat with non-MIRI people present, and nuanced general discussion on that topic happening, is a very different event to have actually happened than the impression this post leaves in the mind of the reader.
Critch and Carey were MIRI people at the time. It wasn’t just them disagreeing with me, I think you and/or Nate were as well.
Do you have a take on Shane Legg? Or any insight to his safety efforts? In his old blog and the XiXiDu interview, he was pretty solid on alignment, back when it was far harder to say such things publicly. And he made this comment in this post, just before starting DeepMind:
I’m even more positive on Shane Legg than Demis Hassabis, but I don’t have the impression he’s in charge.