You’re treating the Michael Druggan quote at the very end as obviously terrible, whereas I see it as obviously sensible. I’m confused. Maybe I’m missing some context? Are you reading in a subtext that Druggan wants the superintelligences to exist, instead of conditioning on the superintelligences existing and talking about what that world would or should be like?
If we assume that the Singularity has happened, and that radical superintelligence exists, and (for the sake of argument) that humans still exist too, … then your position is that humans should still be making consequential decisions about post-Singularity economic policy, legal frameworks, etc.? Really?
Hmm, thinking about it more, I can imagine good-seeming futures in which e.g. there’s a Singleton ASI which enforces hard boundaries (especially against creating other ASIs), but allows lots of human agency within those boundaries (cf. Long Reflection, or Archipeligo, or Nanny AI, or Fun Theory Utopia, etc.). But I wouldn’t exactly call that “humans remain in control”. Or at least, it’s not a central example of that. What other options are there, assuming ASI exists at all?
In any multipolar ASI scenario, the economy and world would presumably be changing at ASI speed, and having excruciatingly slow humans “in control” seems unworkable.
There is a sharp distinction between losing control (even if that doesn’t result in extinction), and delegation without losing control. It’s the distinction between literally permanent disempowerment and the opportunity to grow towards an option to eventually regain control over meaningful resources, where the future of humanity itself becomes superintelligent, at its own pace.
On Twitter, Michael Druggan loudly does not care about x-risk. I could be misremembering but I seem to recall him saying things along the lines of, if ASI kills us then that’s good actually because ASI is the successor species or whatever. His quote sounds worse in that context (where “a non-benevolent one probably just kills us” is followed by an implicit “and that would be a good thing”), but I agree it sounds sensible without context.
Are you reading in a subtext that Druggan wants the superintelligences to exist
That is his stated position IIRC
I went thru his tweets to try to find a direct quote, but he tweets a lot and 95% of it is bodybuilding stuff so I gave up. although I did see some wholesome posts of him complimenting other dudes who were posting their lifting PRs
Druggan: it won’t and that’s ok. We can pass the torch to the new most intelligent species in the known universe
tweeter #2: I would prefer my child to live.
Druggan: Selfish tbh
tweeter #3: But Michael, for you, human replacement and extinction is an upside.
Druggan: To be clear, in my ideal future humans coexist peacefully with ASI . I don’t think extinction is an upside in and of itself. It’s more that I believe that the superintelligence is of such great value its a risk worth taking.
You’re treating the Michael Druggan quote at the very end as obviously terrible, whereas I see it as obviously sensible. I’m confused. Maybe I’m missing some context? Are you reading in a subtext that Druggan wants the superintelligences to exist, instead of conditioning on the superintelligences existing and talking about what that world would or should be like?
If we assume that the Singularity has happened, and that radical superintelligence exists, and (for the sake of argument) that humans still exist too, … then your position is that humans should still be making consequential decisions about post-Singularity economic policy, legal frameworks, etc.? Really?
Hmm, thinking about it more, I can imagine good-seeming futures in which e.g. there’s a Singleton ASI which enforces hard boundaries (especially against creating other ASIs), but allows lots of human agency within those boundaries (cf. Long Reflection, or Archipeligo, or Nanny AI, or Fun Theory Utopia, etc.). But I wouldn’t exactly call that “humans remain in control”. Or at least, it’s not a central example of that. What other options are there, assuming ASI exists at all?
In any multipolar ASI scenario, the economy and world would presumably be changing at ASI speed, and having excruciatingly slow humans “in control” seems unworkable.
There is a sharp distinction between losing control (even if that doesn’t result in extinction), and delegation without losing control. It’s the distinction between literally permanent disempowerment and the opportunity to grow towards an option to eventually regain control over meaningful resources, where the future of humanity itself becomes superintelligent, at its own pace.
On Twitter, Michael Druggan loudly does not care about x-risk. I could be misremembering but I seem to recall him saying things along the lines of, if ASI kills us then that’s good actually because ASI is the successor species or whatever. His quote sounds worse in that context (where “a non-benevolent one probably just kills us” is followed by an implicit “and that would be a good thing”), but I agree it sounds sensible without context.
That is his stated position IIRC
I went thru his tweets to try to find a direct quote, but he tweets a lot and 95% of it is bodybuilding stuff so I gave up. although I did see some wholesome posts of him complimenting other dudes who were posting their lifting PRs
edit: found a quote https://x.com/Michael_Druggan/status/2036464802328093153