Some thoughts: one problem I have with Eliezer’s definition is that bits don’t cost the same in terms of computation because of logical non-omniscience. Imagine two agents in an environment with 2^N possible trajectories corresponding to N bit bitstrings. One agent always outputs the zero bitstring and the other outputs the preimage of some hash function on the zero bitstring or something else expensive like that. Both of these narrow the world down the same amount, and have the same expected influence, but it seems intuitive that if you have to think really hard about each decision you make, then you’re also putting in more optimization in some sense.
Some thoughts: one problem I have with Eliezer’s definition is that bits don’t cost the same in terms of computation because of logical non-omniscience. Imagine two agents in an environment with 2^N possible trajectories corresponding to N bit bitstrings. One agent always outputs the zero bitstring and the other outputs the preimage of some hash function on the zero bitstring or something else expensive like that. Both of these narrow the world down the same amount, and have the same expected influence, but it seems intuitive that if you have to think really hard about each decision you make, then you’re also putting in more optimization in some sense.
Related: Utility Maximization = Description Length Minimization