If we try to translate sentences involving “should” into descriptive sentences about the world, they will probably sound like “action A increases the value of utility function U”.
As you know, there is no commonly agreed upon way of stating “action A increases the value of utility function U” as math (otherwise decision theory would be solved). Given that, what does it mean when I say “I think we should express ‘action A increases the value of utility function U’ in math as X”, which seems like a sensible statement? I don’t see how the “should” in this sentence can be translated into something that sounds like “action A increases the value of utility function U” without making the sentence mean something obviously different.
Given that, what does it mean when I say “I think we should express ‘action A increases the value of utility function U’ in math as X”, which seems like a sensible statement?
I think it makes sense as a statement about decision theories. How would a choice of which mathematical expression of ‘action A increases the value of utility function U’ affect actual utility? Only by affecting which actions are chosen; in other words by selecting a particular (class of) decision theory which maximizes utility due in part to its expression of what “should” means mathematically.
So you’re saying we need to mathematically model our desire for a mathematical model of X? :-) That might be a line of attack, but I don’t yet see what it gives us, compared to attacking the problem directly...
As you know, there is no commonly agreed upon way of stating “action A increases the value of utility function U” as math (otherwise decision theory would be solved). Given that, what does it mean when I say “I think we should express ‘action A increases the value of utility function U’ in math as X”, which seems like a sensible statement? I don’t see how the “should” in this sentence can be translated into something that sounds like “action A increases the value of utility function U” without making the sentence mean something obviously different.
I think it makes sense as a statement about decision theories. How would a choice of which mathematical expression of ‘action A increases the value of utility function U’ affect actual utility? Only by affecting which actions are chosen; in other words by selecting a particular (class of) decision theory which maximizes utility due in part to its expression of what “should” means mathematically.
So you’re saying we need to mathematically model our desire for a mathematical model of X? :-) That might be a line of attack, but I don’t yet see what it gives us, compared to attacking the problem directly...