When I am pondering how a realistic solution to AI existential safety might look like, it actually seems that a realistic solution incorporates a “seed of that positive vision”.
First of all, the “safety properties of the world”must stay invariant as the overall ecosystem rapidly self-modifies. If this is not accomplished, any safety would be short-lived. This invariance cannot be imposed against “natural interests of almost all powerful actors” (attempting to force that would result in something very fragile which is unlikely to hold; the main reason why most attempts to craft existential safety in a world with super-intelligences look so unpromising is that they tend to be fragile in this way).
So one needs the “safety properties of the world” to be in the interests of a sufficiently wide and powerful community of actors, and, for human flourishing, this class of actors should be wide enough to robustly include humans, the class of “entities whose interests are protected” should be crafted in such a way that it includes humans from the very beginning and in such a way that entities cannot be dropped from this class as things evolve.
So, the first approximation to a “realistic solution” looks like a system where all kind of actors (including humans and including powerful AIs) have enough voice to make sure their interests are taken into account, and where the world order which robustly protects those very diverse interests is maintained.
On one hand, powerful AIs have enough incentive to maintain this kind of world order, as no AI can be certain of its relative power in the future and thus needs some protections in light of that future uncertainty. In particular, no one can be safe if it’s OK to drop entities already belonging to the protected class from that protected class. That’s how one makes sure that powerful AIs want to robustly maintain this kind of system, not because of some artificial trick, but because it is very much in the interest of each AI system to do so.
On the other hand, a system like this automatically has a “seed of the future positive vision” simply because it maintains the mechanisms to continue collective deliberations and to continue taking opinions of participants into account. Basically, the positive vision is the future where individuals and groups are not steamrolled, and where they can continue to figure out what they want and need and where these conversations are properly taken into account. We don’t need to decide what those future wants and needs will be, we just need to make sure that the mechanisms to discover those wants and needs and the mechanisms to properly take those discoveries into account continue to work.
When I am pondering how a realistic solution to AI existential safety might look like, it actually seems that a realistic solution incorporates a “seed of that positive vision”.
First of all, the “safety properties of the world”must stay invariant as the overall ecosystem rapidly self-modifies. If this is not accomplished, any safety would be short-lived. This invariance cannot be imposed against “natural interests of almost all powerful actors” (attempting to force that would result in something very fragile which is unlikely to hold; the main reason why most attempts to craft existential safety in a world with super-intelligences look so unpromising is that they tend to be fragile in this way).
So one needs the “safety properties of the world” to be in the interests of a sufficiently wide and powerful community of actors, and, for human flourishing, this class of actors should be wide enough to robustly include humans, the class of “entities whose interests are protected” should be crafted in such a way that it includes humans from the very beginning and in such a way that entities cannot be dropped from this class as things evolve.
So, the first approximation to a “realistic solution” looks like a system where all kind of actors (including humans and including powerful AIs) have enough voice to make sure their interests are taken into account, and where the world order which robustly protects those very diverse interests is maintained.
On one hand, powerful AIs have enough incentive to maintain this kind of world order, as no AI can be certain of its relative power in the future and thus needs some protections in light of that future uncertainty. In particular, no one can be safe if it’s OK to drop entities already belonging to the protected class from that protected class. That’s how one makes sure that powerful AIs want to robustly maintain this kind of system, not because of some artificial trick, but because it is very much in the interest of each AI system to do so.
On the other hand, a system like this automatically has a “seed of the future positive vision” simply because it maintains the mechanisms to continue collective deliberations and to continue taking opinions of participants into account. Basically, the positive vision is the future where individuals and groups are not steamrolled, and where they can continue to figure out what they want and need and where these conversations are properly taken into account. We don’t need to decide what those future wants and needs will be, we just need to make sure that the mechanisms to discover those wants and needs and the mechanisms to properly take those discoveries into account continue to work.