Not all goals have equal fitness, some are self reinforcing. Your goal may be to have great relationships, my goal may be to accumulate money and power. My thing compounds. Reproduction, growth, and propagation feed themselves.
If energy and intelligence are the currency, someone will task an AI system to make more of those things and reinvest.
We could coordinate to suppress those dynamics, but that coordination seems harder in the decentralized futures. I agree that it’s desirable but I don’t see how that world persists.
Do you agree that you’re not describing a stable state? Or is there a stabilizing component that I’m missing?
Looking for examples in nature, you might need something like an immune system auditing nodes in the network to make sure they’re not cancerous. Empathy /deontological tendencies in humans may be another example of a stabilizer. Otherwise a garden without a gardener will become all weeds.
Not all goals have equal fitness, some are self reinforcing. Your goal may be to have great relationships, my goal may be to accumulate money and power. My thing compounds. Reproduction, growth, and propagation feed themselves.
If energy and intelligence are the currency, someone will task an AI system to make more of those things and reinvest.
We could coordinate to suppress those dynamics, but that coordination seems harder in the decentralized futures. I agree that it’s desirable but I don’t see how that world persists.
Do you agree that you’re not describing a stable state? Or is there a stabilizing component that I’m missing?
Looking for examples in nature, you might need something like an immune system auditing nodes in the network to make sure they’re not cancerous. Empathy /deontological tendencies in humans may be another example of a stabilizer. Otherwise a garden without a gardener will become all weeds.