And if we set things up such that the AI’s value system is being decided as a matter of national policy, via a democratic process? Where people discuss it among themselves, with politicians chiming in, and maybe then vote on it, and then the result is interpreted by some bureaucratic process, and then someone types it in? Don’t make me laugh. We’d be lucky if the AI that’d result from this just kills everyone and tiles the universe with paperwork, rather than trapping everyone in some inescapable Kafkaesque nightmare.
(To be clear, it’s not because the people will want bad things. It’s because our processes for eliciting and agglomerating their preferences – any and all processes in wide use – are an abomination.)
We will still need to reconcile our differences later on, of course. But it can be done incrementally, a steady pace of negotiation and power balancing and cultural mingling and sanity-raising. There are routes of intelligence enhancement that are more gradual, that let us preserve this sort of incrementalism and stability while still letting Humanity keep empowering itself. Gradual intelligence augmentation, via biotechnological or cyborgism-like or upload-based means.
The is a lot of statism or pessimism about the potential for improving coordination in your comments. No mentions of the projects of the kind:
Thanks for the links! I’ve been idly thinking about such projects as well, nice to see what ideas others have been considering. Hopefully there’s something workable in there.
But my median prediction is that none of that works, yes. Let alone on the relevant timeline (<10 years). Stuff like Manifold Markets and Twitter’s Community Notes are steps in the right direction, but they’re so very ridiculously tiny. In the meanwhile, the pressures destroying the coordination ability continue to mount.
My optimism would rise dramatically if one of these ideas spawns, say, an SV startup centered around it that ends up valued at billions of dollars within the next three years.
The is a lot of statism or pessimism about the potential for improving coordination in your comments. No mentions of the projects of the kind:
Collective Intelligence Project (+ ecosystem)
AI for Institutions projects (ideas)
Metagov projects, including DAO Science
Gaia Consortium
CLR, FOCAL, AI Objectives Institute
Thanks for the links! I’ve been idly thinking about such projects as well, nice to see what ideas others have been considering. Hopefully there’s something workable in there.
But my median prediction is that none of that works, yes. Let alone on the relevant timeline (<10 years). Stuff like Manifold Markets and Twitter’s Community Notes are steps in the right direction, but they’re so very ridiculously tiny. In the meanwhile, the pressures destroying the coordination ability continue to mount.
My optimism would rise dramatically if one of these ideas spawns, say, an SV startup centered around it that ends up valued at billions of dollars within the next three years.
Maybe this is going to be such a startup? -- A proposal for improving the global online discourse