I’d build satisficers for theoretical reasons, not safety ones. Maximizers to me have problems with modifying/improving the model of the world that they are trying to maximize*. Satisficers don’t tend to use models of the world at the lowest level, instead they get proximate signals for the things they are supposed to me maximizing (e.g. dopamine for evolutionary fitness in animal biology) and have to build models of the world that are good at getting the signals. But they really don’t want to maximise those signals because they are not want they actually are supposed to maximise.
Every time I try to say more than this I lapse into a big long post. I’ll see if I can marshal my thoughts somewhat.
*Things like AIXI don’t have this problem because they don’t have to decide how to best to modify their model as they keep all possible models in mind at once. Which is one reason I don’t think it is a good guide for AI.
I’d build satisficers for theoretical reasons, not safety ones. Maximizers to me have problems with modifying/improving the model of the world that they are trying to maximize*. Satisficers don’t tend to use models of the world at the lowest level, instead they get proximate signals for the things they are supposed to me maximizing (e.g. dopamine for evolutionary fitness in animal biology) and have to build models of the world that are good at getting the signals. But they really don’t want to maximise those signals because they are not want they actually are supposed to maximise.
Every time I try to say more than this I lapse into a big long post. I’ll see if I can marshal my thoughts somewhat.
*Things like AIXI don’t have this problem because they don’t have to decide how to best to modify their model as they keep all possible models in mind at once. Which is one reason I don’t think it is a good guide for AI.