Why does keeping the humans around bolster the strength of their own property rights? If the machines are able to build much better governance than humans have managed, why can’t the new governance regime include a new property system that disappropriates the humans? It’s not like disappropriation is historically novel; humans do it to the losers of wars all the time.
But if the property rights system is a relatively continuous peaceful transition then an eventual regime will struggle with where to draw the line.
Plus, on the way it may be decided to create computational/smart contract governance that cannot be altered and that has control over robots, compute, etc. Yudkowsky envisioned this is “Sysop” or something, a neutral intelligent operating system for the universe. But he got stuck on this “decisive action AKA take over the world” as a prerequisite, gave up and became pro-pause. But I think that was a mistake.
Owning shares in most modern companies won’t be useful in sufficiently distant future, and might prove insufficient to pay for survival. Even that could be eaten away by dilution, over astronomical time. The reachable universe is not a growing pie, ability to reinvest into relevant entities won’t necessarily be open.
Owning shares in most modern companies won’t be useful in sufficiently distant future, and might prove insufficient to pay for survival
Well there may simply be better index funds. In fact QQQ is already pretty good.
The insight is that better property rights are both positive for AI civilization (whether the owners are AIs, humans, uplifted dolphins, etc) and also better for normie legacy humans.
It is not a battle of humans vs AIs, but rather of order (strong property rights, good solutions to game theory) versus chaos (weak property rights, burning of the cosmic commons, bad equilibria).
I think the “order vs chaos not humans vs AIs”, “we (AIs, humans) are all on team order” is an underrated perspective.
Why does keeping the humans around bolster the strength of their own property rights? If the machines are able to build much better governance than humans have managed, why can’t the new governance regime include a new property system that disappropriates the humans? It’s not like disappropriation is historically novel; humans do it to the losers of wars all the time.
Well if there was a violent takeover, yes.
But if the property rights system is a relatively continuous peaceful transition then an eventual regime will struggle with where to draw the line.
Plus, on the way it may be decided to create computational/smart contract governance that cannot be altered and that has control over robots, compute, etc. Yudkowsky envisioned this is “Sysop” or something, a neutral intelligent operating system for the universe. But he got stuck on this “decisive action AKA take over the world” as a prerequisite, gave up and became pro-pause. But I think that was a mistake.
Owning shares in most modern companies won’t be useful in sufficiently distant future, and might prove insufficient to pay for survival. Even that could be eaten away by dilution, over astronomical time. The reachable universe is not a growing pie, ability to reinvest into relevant entities won’t necessarily be open.
Well there may simply be better index funds. In fact QQQ is already pretty good.
The insight is that better property rights are both positive for AI civilization (whether the owners are AIs, humans, uplifted dolphins, etc) and also better for normie legacy humans.
It is not a battle of humans vs AIs, but rather of order (strong property rights, good solutions to game theory) versus chaos (weak property rights, burning of the cosmic commons, bad equilibria).
I think the “order vs chaos not humans vs AIs”, “we (AIs, humans) are all on team order” is an underrated perspective.