And throughout, the prevailing meme was “don’t tell the world about AGI, we don’t want governments to catch on, they’ll probably do something dumb. Better keep this to the right kind of smart technical people.” A view that only changed in the past couple of years.
Does pushing for a lot of public fear about this kind of research, that makes all projects hard, seem hopeless?
Eliezer Yudkowsky
What does it buy us? 3 months of delay at the cost of a tremendous amount of goodwill? 2 years of delay? What’s that delay for, if we all die at the end? Even if we then got a technical miracle, would it end up impossible to run a project that could make use of an alignment miracle, because everybody was afraid of that project? Wouldn’t that fear tend to be channeled into “ah, yes, it must be a government project, they’re the good guys” and then the government is much more hopeless and much harder to improve upon than Deepmind?
Perhaps I am just distinguishing between two lizards rather than measuring absolutely, but at least Eliezer/MIRI were willing to talk about these things openly and in public. This is just them not (yet) going down the route of public advocacy, whereas (as I say in my other comment) the OpenPhil EAs required confidentiality before they’d talk seriously about timelines, I think until ChatGPT.
This one seems inaccurate to me:
Eliezer was the main person I was able to get useful public thinking about timelines from, writing in 2017 about both AlphGo Zero and the Foom Debate and There’s No Fire Alarm for Artificial General Intelligence, the latter of which remains one of the most influential things on my own decision-making. Whereas the OpenPhil EAs were the ones who I was explicitly required to agree to confidentiality, and told not to write about timelines, as it would cause the rest of the world to act stupidly and race.
from https://intelligence.org/2021/11/11/discussion-with-eliezer-yudkowsky-on-agi-interventions/ (this is the closet thing Perplexity found)
Anonymous
Does pushing for a lot of public fear about this kind of research, that makes all projects hard, seem hopeless?
Eliezer Yudkowsky
What does it buy us? 3 months of delay at the cost of a tremendous amount of goodwill? 2 years of delay? What’s that delay for, if we all die at the end? Even if we then got a technical miracle, would it end up impossible to run a project that could make use of an alignment miracle, because everybody was afraid of that project? Wouldn’t that fear tend to be channeled into “ah, yes, it must be a government project, they’re the good guys” and then the government is much more hopeless and much harder to improve upon than Deepmind?
Perhaps I am just distinguishing between two lizards rather than measuring absolutely, but at least Eliezer/MIRI were willing to talk about these things openly and in public. This is just them not (yet) going down the route of public advocacy, whereas (as I say in my other comment) the OpenPhil EAs required confidentiality before they’d talk seriously about timelines, I think until ChatGPT.