I’ve long been somewhat skeptical that utility functions are the right abstraction.
My argument is also rather handwavy, being something like “this is the wrong abstraction for how agents actually function, so even if you can always construct a utility function and say some interesting things about its properties, it doesn’t tell you the thing you need to know to understand and predict how an agent will behave”. In my mind I liken it to the state of trying to code in functional programming languages on modern computers: you can do it, but you’re also fighting an uphill battle against the way the computer is physically implemented, so don’t be surprised if things get confusing.
And much like in the utility function case, people still program in functional languages because of the benefits they confer. I think the same is true of utility functions: they confer some big benefits when trying to reason about certain problems, so we accept the tradeoffs of using them. I think that’s fine so long as we have a morphism to other abstractions that will work better for understanding the things that utility functions obscure.
I’ve long been somewhat skeptical that utility functions are the right abstraction.
My argument is also rather handwavy, being something like “this is the wrong abstraction for how agents actually function, so even if you can always construct a utility function and say some interesting things about its properties, it doesn’t tell you the thing you need to know to understand and predict how an agent will behave”. In my mind I liken it to the state of trying to code in functional programming languages on modern computers: you can do it, but you’re also fighting an uphill battle against the way the computer is physically implemented, so don’t be surprised if things get confusing.
And much like in the utility function case, people still program in functional languages because of the benefits they confer. I think the same is true of utility functions: they confer some big benefits when trying to reason about certain problems, so we accept the tradeoffs of using them. I think that’s fine so long as we have a morphism to other abstractions that will work better for understanding the things that utility functions obscure.