Mostly agree with the content of this post, though I had some trouble with the Alice & Rick part.
I buy that the active chains-of-thought/actions in Alice’s mind will typically be ones that aren’t objected to by her current shards, both those that implement her Rick-approval-seeking and those that implement her aversion-to-converting. If they had been significantly objected to by those shards, the relevant thoughts & actions would have been discarded in favor of other ones. Given that, it makes sense to me that when you eliminate those, the remaining courses of action still leading to Alice converting might mainly involve “slow”/covert value drift.
What I am uncertain about is the appropriate level of agency to ascribe to her Rick-approval shard etc. I don’t personally imagine that such a shard is foresightedly (even if non-introspectively) searching for plans in the way described (for example, not bidding up the direct conversion plan if it were considered); any search it does would be much more passive. Like perhaps the behavior of the bundle of circuits could be “whenever the Rick-approval node turns off, make Alice feel a sharp pang of longing”, and “when Alice thinks of something that lights up the Rick-approval node, bid for it no matter how irrational the thought is”, and maybe even “when the Rick-approval node lights up, trigger the general-purpose rationalization machinery in the language area to start babbling”. But I imagine that the heavy lifting & sophistication is largely outside the shard.
Mostly agree with the content of this post, though I had some trouble with the Alice & Rick part.
I buy that the active chains-of-thought/actions in Alice’s mind will typically be ones that aren’t objected to by her current shards, both those that implement her Rick-approval-seeking and those that implement her aversion-to-converting. If they had been significantly objected to by those shards, the relevant thoughts & actions would have been discarded in favor of other ones. Given that, it makes sense to me that when you eliminate those, the remaining courses of action still leading to Alice converting might mainly involve “slow”/covert value drift.
What I am uncertain about is the appropriate level of agency to ascribe to her Rick-approval shard etc. I don’t personally imagine that such a shard is foresightedly (even if non-introspectively) searching for plans in the way described (for example, not bidding up the direct conversion plan if it were considered); any search it does would be much more passive. Like perhaps the behavior of the bundle of circuits could be “whenever the Rick-approval node turns off, make Alice feel a sharp pang of longing”, and “when Alice thinks of something that lights up the Rick-approval node, bid for it no matter how irrational the thought is”, and maybe even “when the Rick-approval node lights up, trigger the general-purpose rationalization machinery in the language area to start babbling”. But I imagine that the heavy lifting & sophistication is largely outside the shard.