I think we know of at least one working idea of the utility function for one individual human in theory—show the human all the possible simulated futures and allow them to pick the best.
Of course that would be a complex vector utility function, and you couldn’t hope to map that down to a single simple scalar value that aggregates (even with voting).
The true optimal theoretical ‘utility function’ for humanity as a whole is probably something akin to simulating the evolution of human culture/civilization forward into many possible futures, disseminating all that knowledge, feeding that back in on itself recursively with numerous rounds of re-simulation and then allowing current political/economic systems to evolve forward with all that processed future-path knowledge.
I think we know of at least one working idea of the utility function for one individual human in theory—show the human all the possible simulated futures and allow them to pick the best.
Of course that would be a complex vector utility function, and you couldn’t hope to map that down to a single simple scalar value that aggregates (even with voting).
The true optimal theoretical ‘utility function’ for humanity as a whole is probably something akin to simulating the evolution of human culture/civilization forward into many possible futures, disseminating all that knowledge, feeding that back in on itself recursively with numerous rounds of re-simulation and then allowing current political/economic systems to evolve forward with all that processed future-path knowledge.