The current trajectory of AI development is reckless and we need more time.
But it’s way more complicated than that.
It seems like this all ultimately becomes a matter of designing an effective, robust, future-proof institution aimed at guiding humanity to a good future. Similar to what the Founding Fathers attempted with the United States, but with enough foresight to account for huge and unpredictable changes in technological capability.
And then, a matter of getting powerful people on-board, which probably means showing everyone that there are a million ways for humanity to destroy itself[1] if our process of navigating the future isn’t done with extreme care.
Why can’t we start by agreeing on this relatively simple message? Is there some way to get the AI community to at least come together here, despite all other disagreements? Does it not greatly frustrate everyone else whenever important AI-leaders recycle the same vague oversimplifications[2] of what is literally humanity’s ultimate problem?
I feel like I’m going a little crazy here. Everyone gestures at different research directions they find promising but it seems like everything eventually falls apart[3] when you recognize that we are on a planet run by apes that will soon wield self-destructive power. At the end of the day, effective political governance and coordination are a pre-requisite to the survival of our species and it does seem like this is incredibly neglected.
Perhaps there is hope if frontier labs, the members of which may become extremely wealthy and powerful, start actively planning for and prioritizing this process of designing–with much deliberation and careful philosophical consideration–the successor governance system for the human race. What can we do to make that more likely?
Dario says “Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.” And: “The years in front of us will be impossibly hard, asking more of us than we think we can give.”
So, what are we doing about that? Have there been any attempts at creating or investing in institutions that try to figure this sort of thing out? As far as I can tell, AI governance is a fragmented, not-very-rigorous field of study which just seems completely backwards, and something that we could at least be trying way harder to solve.
Including alignment. For God’s sake, what/who are we even aligning AI to? The world still ends if everyone has an obedient super-intelligence in their pocket.
What I’m gathering from this is:
The current trajectory of AI development is reckless and we need more time.
But it’s way more complicated than that.
It seems like this all ultimately becomes a matter of designing an effective, robust, future-proof institution aimed at guiding humanity to a good future. Similar to what the Founding Fathers attempted with the United States, but with enough foresight to account for huge and unpredictable changes in technological capability.
And then, a matter of getting powerful people on-board, which probably means showing everyone that there are a million ways for humanity to destroy itself[1] if our process of navigating the future isn’t done with extreme care.
Why can’t we start by agreeing on this relatively simple message? Is there some way to get the AI community to at least come together here, despite all other disagreements? Does it not greatly frustrate everyone else whenever important AI-leaders recycle the same vague oversimplifications[2] of what is literally humanity’s ultimate problem?
I feel like I’m going a little crazy here. Everyone gestures at different research directions they find promising but it seems like everything eventually falls apart[3] when you recognize that we are on a planet run by apes that will soon wield self-destructive power. At the end of the day, effective political governance and coordination are a pre-requisite to the survival of our species and it does seem like this is incredibly neglected.
Perhaps there is hope if frontier labs, the members of which may become extremely wealthy and powerful, start actively planning for and prioritizing this process of designing–with much deliberation and careful philosophical consideration–the successor governance system for the human race. What can we do to make that more likely?
We probably do have to worry about cultural drift and other “future issues” quickly made salient by AI-driven acceleration.
Dario says “Humanity is about to be handed almost unimaginable power, and it is deeply unclear whether our social, political, and technological systems possess the maturity to wield it.” And: “The years in front of us will be impossibly hard, asking more of us than we think we can give.”
So, what are we doing about that? Have there been any attempts at creating or investing in institutions that try to figure this sort of thing out? As far as I can tell, AI governance is a fragmented, not-very-rigorous field of study which just seems completely backwards, and something that we could at least be trying way harder to solve.
Including alignment. For God’s sake, what/who are we even aligning AI to? The world still ends if everyone has an obedient super-intelligence in their pocket.