I think the simplifying features on the other side outweigh those—ie., it’s built from atomic units that do exactly what you tell them to, and there are probably fewer abstraction layers between those atomic units and the goal system. But I do think Mechanism Design is an important field, and will probably form an important part of any friendly optimizing process.
Organisations are likely to build machine intelligence and imbue it with their values. That is reason enough to be concerned with organisation values. One of my proposals to help with this is better corporate repuatation systems.
I think the simplifying features on the other side outweigh those—ie., it’s built from atomic units that do exactly what you tell them to, and there are probably fewer abstraction layers between those atomic units and the goal system. But I do think Mechanism Design is an important field, and will probably form an important part of any friendly optimizing process.
Organisations are likely to build machine intelligence and imbue it with their values. That is reason enough to be concerned with organisation values. One of my proposals to help with this is better corporate repuatation systems.