To elaborate on “the preferences of the agent are built in”, that means the agent is coded with a description of a large but fixed mathematical formula with no free variables, and wants the value of that formula to be as high as possible. That doesn’t make much sense in simple cases like “I want the value of 2+2 to be as high as possible”, but it works in more complicated cases where the formula contains instances of the agent itself, which is possible by quining.
To elaborate on why “scanning each universe model for structures that will be logically dependent on its output” doesn’t need bridging laws, let’s note that it can be viewed as theorem proving. The agent might look for easily provable theorems of the form “if my mathematical structure has a certain input-output map, then this particular universe model returns a certain value”. Or it could use some kind of approximate logical reasoning, but in any case it wouldn’t need explicit bridging laws.
Yeah, your explanation sounds right.
To elaborate on “the preferences of the agent are built in”, that means the agent is coded with a description of a large but fixed mathematical formula with no free variables, and wants the value of that formula to be as high as possible. That doesn’t make much sense in simple cases like “I want the value of 2+2 to be as high as possible”, but it works in more complicated cases where the formula contains instances of the agent itself, which is possible by quining.
To elaborate on why “scanning each universe model for structures that will be logically dependent on its output” doesn’t need bridging laws, let’s note that it can be viewed as theorem proving. The agent might look for easily provable theorems of the form “if my mathematical structure has a certain input-output map, then this particular universe model returns a certain value”. Or it could use some kind of approximate logical reasoning, but in any case it wouldn’t need explicit bridging laws.