Rogue AI won’t necessarily have to resort to crime though. It might make money legitimately, like by doing freelance work, or running a SaaS business. I expect these cases would be a lot harder to detect.
Steven McCulloch
Karma: 64
It All Started With a Mac Mini
The Inevitable Evolution of AI Agents
I’m really surprised that so few people in the AI community have heard of metacrisis and Daniel Schmactenberger, since the ideas are rooted in game theory, moloch, coordination problems, evolutionary dynamics, etc… I think it’s really useful to look at these big problems through multiple lenses, and the metacrisis lens is a fresh and underexplored perspective that I think more people in this community should explore further.
Steven McCulloch’s Shortform
I’d like to introduce LessWrong to Daniel Schmactenberger, a social philosopher with a focus on existential risk, Moloch, game theory, and the metacrisis. In my opinion, a hidden gem, and vastly under-appreciated. I recommend listening to his appearances on podcasts & public talks. He has a few great interviews on Nate Hagen’s channel. My favorite is this:
I think he has a fresh and valuable perspective because he is totally not a rationalist, not technically inclined, and yet he understands the risks of AI and takes the risks very seriously. He cares a lot about making the future go well, and solving coordination problems. He talks often about avoiding catastrophes, or dystopias, and instead finding a ‘third attractor’ - something positive-sum that also out-competes reckless power-grabbing. I think listening to him can spark insights because he uses entirely different language, arguments, and mental models than the ones that are common in rationalist spaces.
If you prefer to read, here’s an essay of his I recommend: https://consilienceproject.org/technology-is-not-values-neutral-ending-the-reign-of-nihilistic-design-2/
I don’t have much of my own personal insight to add here, but I think simply connecting smart people with other smart people can be pretty high leverage. Would love to hear if anyone else has been influenced by him.
This was an interesting watch. Just a few days ago
Challenges Posed by Artificial Intelligence and its Regulation
Witnesses
As an individual • Steven Adler, Artificial Intelligence Researcher The Human Line Project • Etienne Brisson, Chief Executive Officer ControlAI • Andrea Miotti, Chief Executive Officer AI Governance and Safety Canada • Wyatt Tessari L’Allié, Founder and Executive Director
It’s inspiring to watch these people saying the right things to the right people.
https://www.ourcommons.ca/committees/en/WitnessMeetings?organizationId=43696