Maybe add “[Link]” to the title to appease the people who get angry about that sort of stuff.
That planning is not a viable alternative to capitalism (as opposed to a tool within it) should disturb even capitalism’s most ardent partisans. It means that their system faces no competition, nor even any plausible threat of competition. Those partisans themselves should be able to say what will happen then: the masters of the system, will be tempted, and more than tempted, to claim more and more of what it produces as monopoly rents. This does not end happily.
A consideration like this seems to demand a refinement of the concept of FAI. To what extent is an AI that puts itself on the top of the existing economic order and extracts rents (maybe more, maybe less than the folks currently on top) friendly or unfriendly? Given the complexity and information-gathering constraints involved in economic planning, an we expect an AI to aim for anything more/better than this?
it says nothing about whether it is friendly or unfriendly. If it extracts rents from things humans don’t actually want in reflective equilibrium and channels it into things they do want it is friendly, vice versa unfriendly.
it is also phenomenally unlikely that the most powerful optimization an AI will be able to do is piggyback our financial system.
Maybe add “[Link]” to the title to appease the people who get angry about that sort of stuff.
A consideration like this seems to demand a refinement of the concept of FAI. To what extent is an AI that puts itself on the top of the existing economic order and extracts rents (maybe more, maybe less than the folks currently on top) friendly or unfriendly? Given the complexity and information-gathering constraints involved in economic planning, an we expect an AI to aim for anything more/better than this?
it says nothing about whether it is friendly or unfriendly. If it extracts rents from things humans don’t actually want in reflective equilibrium and channels it into things they do want it is friendly, vice versa unfriendly.
it is also phenomenally unlikely that the most powerful optimization an AI will be able to do is piggyback our financial system.