I’m really enjoying all these posts, thanks a lot!
something about the argument brought unpleasant emotions into your mind. A subsystem activated with the goal of making those emotions go away, and an effective way of doing so was focusing your attention on everything that could be said to be wrong with your spouse.
Wouldn’t it be simpler to say that righteous indignation is a rewarding feeling (in the moment) and we’re motivated to think thoughts that bring about that feeling?
the “regretful subsystem” cannot directly influence the “nasty subsystem”
Agreed, and this is one of the reasons that I think normal intuitions about how agents behave don’t necessarily carry over to self-modifying agents whose subagents can launch direct attacks against each other, see here.
it looks like craving subsystems run on cached models
Yeah, just like every other subsystem right? Whenever any subsystem (a.k.a. model a.k.a. hypothesis) gets activated, it turns on a set of associated predictions. If it’s a model that says “that thing in my field of view is a cat”, it activates some predictions about parts of the visual field. If it’s a model that says “I am going to brush my hair in a particular way”, it activates a bunch of motor control commands and related sensory predictions. If it’s a model that says “I am going to get angry at them”, it activates, um, hormones or something, to bring about the corresponding arousal and valence. All these examples seem like the same type of thing to me, and all of them seem like “cached models”.
I’m really enjoying all these posts, thanks a lot!
Thank you for saying that. :)
Wouldn’t it be simpler to say that righteous indignation is a rewarding feeling (in the moment) and we’re motivated to think thoughts that bring about that feeling?
Well, in my model there are two layers:
1) first, the anger is produced by a subsystem which is optimizing for some particular goal
2) if that anger looks like it is achieving the intended goal, then positive valence is produced as a result; that is experienced as a rewarding feeling that e.g. craving may grab hold of and seek to maintain
That said, the exact reason for why anger is produced isn’t really important for the example and might just be unnecessarily distracting, so I’ll remove it.
Agreed, and this is one of the reasons that I think normal intuitions about how agents behave don’t necessarily carry over to self-modifying agents whose subagents can launch direct attacks against each other, see here.
Agreed.
Yeah, just like every other subsystem right?
Yep! Well, some subsystems seem to do actual forward planning as well, but of course that planning is based on cached models.
I’m really enjoying all these posts, thanks a lot!
Wouldn’t it be simpler to say that righteous indignation is a rewarding feeling (in the moment) and we’re motivated to think thoughts that bring about that feeling?
Agreed, and this is one of the reasons that I think normal intuitions about how agents behave don’t necessarily carry over to self-modifying agents whose subagents can launch direct attacks against each other, see here.
Yeah, just like every other subsystem right? Whenever any subsystem (a.k.a. model a.k.a. hypothesis) gets activated, it turns on a set of associated predictions. If it’s a model that says “that thing in my field of view is a cat”, it activates some predictions about parts of the visual field. If it’s a model that says “I am going to brush my hair in a particular way”, it activates a bunch of motor control commands and related sensory predictions. If it’s a model that says “I am going to get angry at them”, it activates, um, hormones or something, to bring about the corresponding arousal and valence. All these examples seem like the same type of thing to me, and all of them seem like “cached models”.
Thank you for saying that. :)
Well, in my model there are two layers:
1) first, the anger is produced by a subsystem which is optimizing for some particular goal
2) if that anger looks like it is achieving the intended goal, then positive valence is produced as a result; that is experienced as a rewarding feeling that e.g. craving may grab hold of and seek to maintain
That said, the exact reason for why anger is produced isn’t really important for the example and might just be unnecessarily distracting, so I’ll remove it.
Agreed.
Yep! Well, some subsystems seem to do actual forward planning as well, but of course that planning is based on cached models.