That is, of course, untrue. The human brain might contain useless noise.
But how much simpler would utility be? What if it is 10^15 operations per moral decision (i mean, moral comparison between two worlds). Good luck using it to process different choices for a write-in problem.
Indeed, it seems quite likely that the human utility function is essentially coded in the human genome.
Why not in the very laws of universe, at that point? The DNA is not blueprint, it’s a recipe, and it does not contain any of our culture.
edit:
To reiterate the intended point, the idea that “Any computable agent may described using a utility function” illustrates that the idea of building a programmable intelligent machine—and then plugging a utility function into it—is quite a general one.
None of that. For agents that don’t implement maximization of simple utility, the utility function ‘description’ which was mathematically proven, includes complete copy of the agent and you gain nothing what so ever by plugging it into some utility maximizer. You just have the maximizer relay the agent’s actions, without doing anything useful.
Indeed, it seems quite likely that the human utility function is essentially coded in the human genome.
Why not in the very laws of universe, at that point? [...]
That is not totally impossible. The universe does seem to have some “magic numbers”—which we can’t currently explain and which contain significant complexity. The “fine structure constant” for example. In principle, the billionth digit of this could contain useful information about the human utility function, expressed via the wonders of the way chaos theory enables small changes to make big differences.
However, one has to ask: how plausible this is. More likely that the physical constants are not critical beyond a few dozen decimal places. In which case, the laws of the universe look as though they are probably effectively small—and then the human utility function seems unlikely to fit into a description of them.
The point is that laws of the universe, lead to humans, via repeated application of those laws. DNA, too, leads to humans, via repeated use of that DNA (and above-mentioned laws of physics), but combined with the environment and culture. I don’t sure that we would like the raw human utility function, sans culture, to be used for any sort of decisions. There’s no good reason to expect the results to be nice, given just how many screwed up things other cultures did (look at Aztec)
In any case, calculating the human utility from DNA, given that DNA is not a blueprint, would involve embryonic development simulation followed by brain simulation.
But how much simpler would utility be? What if it is 10^15 operations per moral decision (i mean, moral comparison between two worlds). Good luck using it to process different choices for a write-in problem.
Why not in the very laws of universe, at that point? The DNA is not blueprint, it’s a recipe, and it does not contain any of our culture.
edit:
None of that. For agents that don’t implement maximization of simple utility, the utility function ‘description’ which was mathematically proven, includes complete copy of the agent and you gain nothing what so ever by plugging it into some utility maximizer. You just have the maximizer relay the agent’s actions, without doing anything useful.
That is not totally impossible. The universe does seem to have some “magic numbers”—which we can’t currently explain and which contain significant complexity. The “fine structure constant” for example. In principle, the billionth digit of this could contain useful information about the human utility function, expressed via the wonders of the way chaos theory enables small changes to make big differences.
However, one has to ask: how plausible this is. More likely that the physical constants are not critical beyond a few dozen decimal places. In which case, the laws of the universe look as though they are probably effectively small—and then the human utility function seems unlikely to fit into a description of them.
The point is that laws of the universe, lead to humans, via repeated application of those laws. DNA, too, leads to humans, via repeated use of that DNA (and above-mentioned laws of physics), but combined with the environment and culture. I don’t sure that we would like the raw human utility function, sans culture, to be used for any sort of decisions. There’s no good reason to expect the results to be nice, given just how many screwed up things other cultures did (look at Aztec)
I don’t deny that culture has an influence over what humans want. That’s part of what got the “essentially” put into my statement—and emphasised.
In any case, calculating the human utility from DNA, given that DNA is not a blueprint, would involve embryonic development simulation followed by brain simulation.