The general problem seems difficult. There are plenty of ways to model satisficing behavior.
Suppose we are given a utility function u. If we have some information about the maximum, here are some possibilities:
The satisficer follows some maximization algorithm and terminates once some threshold is attained (say, 90% of max(u)).
The satisficer uses a probabilistic algorithm that terminates after a certain threshold is probably attained.
The satisficer has some threshold, but continues searching as long as it’s “cheap” to do so, where “cheap” depends on some auxiliary cost function. It could, for example, measure the number of steps since attaining the best known value thus far or it could measure the rate of improvement and stop once that dips too low.
If we don’t know much about the maximum, we can’t very well have a predefined threshold, but there some strategies:
The satisficer follows some algorithm for maximizing u and terminates after, say, N steps and chooses the best value found. Or one can add some adaptive behavior and say that it terminates if there has been no improvement in a certain time interval.
In a constrained problem where searching is expensive, the satisficer accepts the first solution satisfying the constraints.
What the above have in common is that the satisficer is basically carrying out some optimization algorithm with alternative stopping conditions. As such, it may exhibit some of the drawbacks one finds with optimizers. Maybe there’s a model that’s not built on an optimizer that I haven’t thought of.
This contradict the intuitive picture that many people have of a satisficer, which is an agent that does the minimum of effort to reach its goal, and doesn’t mess up the outside world more than it has to. And if it can’t accomplish the goals without messing up the outside world, it would be content not to.
There’s a tacit assumption here that messing with the outside world involves effort. Without it, you can get all kinds of unexpected behavior. If the satisficer is fine with “make 10 paperclips”, but the difference between that and “make 10 paperclips and kill all humans” is small, the satisficer may well go with whatever showed up first in its search algorithm.
That’s just a relic of mathematics though. If you’re trying to find the maximum of, say, the size of the projection of some 10-dimensional set on a particular 2-dimensional plane, there may be a lot of room in those extra dimensions that doesn’t impact the actual value to be maximized. Trying to find something merely close to the max would seem to exacerbate this issue.
If the satisficer is fine with “make 10 paperclips”, but the difference between that and “make 10 paperclips and kill all humans” is small, the satisficer may well go with whatever showed up first in its search algorithm.
Yep, that’s the problem. That’s why I’m trying to address the issue directly by penalising things like ”...and kill all humans”.
The general problem seems difficult. There are plenty of ways to model satisficing behavior.
Suppose we are given a utility function u. If we have some information about the maximum, here are some possibilities:
The satisficer follows some maximization algorithm and terminates once some threshold is attained (say, 90% of max(u)).
The satisficer uses a probabilistic algorithm that terminates after a certain threshold is probably attained.
The satisficer has some threshold, but continues searching as long as it’s “cheap” to do so, where “cheap” depends on some auxiliary cost function. It could, for example, measure the number of steps since attaining the best known value thus far or it could measure the rate of improvement and stop once that dips too low.
If we don’t know much about the maximum, we can’t very well have a predefined threshold, but there some strategies:
The satisficer follows some algorithm for maximizing u and terminates after, say, N steps and chooses the best value found. Or one can add some adaptive behavior and say that it terminates if there has been no improvement in a certain time interval.
In a constrained problem where searching is expensive, the satisficer accepts the first solution satisfying the constraints.
What the above have in common is that the satisficer is basically carrying out some optimization algorithm with alternative stopping conditions. As such, it may exhibit some of the drawbacks one finds with optimizers. Maybe there’s a model that’s not built on an optimizer that I haven’t thought of.
There’s a tacit assumption here that messing with the outside world involves effort. Without it, you can get all kinds of unexpected behavior. If the satisficer is fine with “make 10 paperclips”, but the difference between that and “make 10 paperclips and kill all humans” is small, the satisficer may well go with whatever showed up first in its search algorithm.
That’s just a relic of mathematics though. If you’re trying to find the maximum of, say, the size of the projection of some 10-dimensional set on a particular 2-dimensional plane, there may be a lot of room in those extra dimensions that doesn’t impact the actual value to be maximized. Trying to find something merely close to the max would seem to exacerbate this issue.
Yep, that’s the problem. That’s why I’m trying to address the issue directly by penalising things like ”...and kill all humans”.