It’s a choice what level of independence or autonomy to give to agents. I think we can make the choice of a human centered world, where the economy remains about what humans value. And as more of their material needs are satisfied, what humans value might move more towards relations with other humans.
My point is that there is single objective function that corresponds to “making the economy better”- humans get to choose their own objectives. Yes, if our goal was to advance on the Kardashev scale as fast as possible, and maximize energy production, then it might make sense to just hand over control to the AIs. But we don’t have to make it our goal.
Not all goals have equal fitness, some are self reinforcing. Your goal may be to have great relationships, my goal may be to accumulate money and power. My thing compounds. Reproduction, growth, and propagation feed themselves.
If energy and intelligence are the currency, someone will task an AI system to make more of those things and reinvest.
We could coordinate to suppress those dynamics, but that coordination seems harder in the decentralized futures. I agree that it’s desirable but I don’t see how that world persists.
Do you agree that you’re not describing a stable state? Or is there a stabilizing component that I’m missing?
Looking for examples in nature, you might need something like an immune system auditing nodes in the network to make sure they’re not cancerous. Empathy /deontological tendencies in humans may be another example of a stabilizer. Otherwise a garden without a gardener will become all weeds.
It’s a choice what level of independence or autonomy to give to agents. I think we can make the choice of a human centered world, where the economy remains about what humans value. And as more of their material needs are satisfied, what humans value might move more towards relations with other humans.
My point is that there is single objective function that corresponds to “making the economy better”- humans get to choose their own objectives. Yes, if our goal was to advance on the Kardashev scale as fast as possible, and maximize energy production, then it might make sense to just hand over control to the AIs. But we don’t have to make it our goal.
Not all goals have equal fitness, some are self reinforcing. Your goal may be to have great relationships, my goal may be to accumulate money and power. My thing compounds. Reproduction, growth, and propagation feed themselves.
If energy and intelligence are the currency, someone will task an AI system to make more of those things and reinvest.
We could coordinate to suppress those dynamics, but that coordination seems harder in the decentralized futures. I agree that it’s desirable but I don’t see how that world persists.
Do you agree that you’re not describing a stable state? Or is there a stabilizing component that I’m missing?
Looking for examples in nature, you might need something like an immune system auditing nodes in the network to make sure they’re not cancerous. Empathy /deontological tendencies in humans may be another example of a stabilizer. Otherwise a garden without a gardener will become all weeds.