It’s like the old “dragon in the garage” parable: the woman is too good at systematically denying the things which would actually help to not have a working model somewhere in there
I think you’re still imagining too coherent an agent. Yes, perhaps there is a slice through her mind that contains a working model which, if that model were dropped into the mind of a more coherent agent, could be used to easily comprehend and fix the situation. But this slice doesn’t necessarily have executive conscious control at any given moment, and if it ever does, it isn’t necessarily the same slice that contains her baseline/reflectively endorsed personality.
E. g., perhaps, at any given moment, only part of that model is visible to the conscious mind, a 3D object sliding through a 2D plane, and the person can’t really take in the whole of it at once, realize how ridiculous they’re being, and act on it rationally. Or perhaps the thought of confronting the problem causes overwhelming distress due to malfunctioning emotional circuitry, and so do the thoughts of fixing that circuitry, in a way that recurses on itself indefinitely/in the style of TD learning. Or something else that’s just as messy.
Human brains haven’t solved self-interpretability, and human minds aren’t arranged into even the approximate shape of coherent agents by default. Just because there’s a module in there somewhere which implements a model of X doesn’t mean the person can casually reach in and do things with that module.
Edit: After reading your other responses, yeah, “not modeling them as creatures particularly similar to yourself” might just be the correct approach. I, uh, also don’t find people with weak metacognitive skills particularly relatable.
In my mental ontology, there’s a set of specific concepts and mental motions associated with accountability: viewing people as being responsible for their actions, being disappointed in or impressed by their choices, modeling the assignment of blame/credit as meaningful operations. Implicitly, this requires modeling other people as agents: types of systems which are usefully modeled as having control over their actions. To me, this is a prerequisite for being able to truly connect with someone.
When you apply the not-that-coherent-an-agent lens, you do lose that. Because, like, which parts of that person’s cognition should you interpret as the agent making choices, and which as parts of the malfunctioning exoskeleton the agent has no control over? You can make some decision about that, but this is usually pretty arbitrary. If someone is best modeled like this, they’re not well-modeled as an agent, and holding them accountable is a category error. They’re a type of system that does what it does.
You can still invoke the social rituals of “blame” and “responsibility” if you expect that to change their behavior, but the mental experience of doing so is very different. It’s more like calculating the nudges you need to make to prompt the desired mechanistic behavior, rather than as interfacing with a fellow person. In the latter case, you can sort of relax, communicate in a way focused on transferring information, instead of focusing on the form of communication, and trust them to make correct inferences. In the former case, you need to keep precise track of tone/wording/aesthetics/etc., and it’s less “communication” and more “optimization”.
I really dislike thinking of people in this way, and I try to adopt the viewing-them-as-a-person frame whenever it’s at all possible. But the other frame does unfortunately seem to be useful in many cases. Trying to do otherwise often feels like reaching out for someone’s hand and finding nothing there.
If this is what you meant by viewing others as cats, yeah, that tracks.
@Caleb Biddulph’s reply seems right to me. Another tack:
I think you’re still imagining too coherent an agent. Yes, perhaps there is a slice through her mind that contains a working model which, if that model were dropped into the mind of a more coherent agent, could be used to easily comprehend and fix the situation. But this slice doesn’t necessarily have executive conscious control at any given moment, and if it ever does, it isn’t necessarily the same slice that contains her baseline/reflectively endorsed personality.
E. g., perhaps, at any given moment, only part of that model is visible to the conscious mind, a 3D object sliding through a 2D plane, and the person can’t really take in the whole of it at once, realize how ridiculous they’re being, and act on it rationally. Or perhaps the thought of confronting the problem causes overwhelming distress due to malfunctioning emotional circuitry, and so do the thoughts of fixing that circuitry, in a way that recurses on itself indefinitely/in the style of TD learning. Or something else that’s just as messy.
Human brains haven’t solved self-interpretability, and human minds aren’t arranged into even the approximate shape of coherent agents by default. Just because there’s a module in there somewhere which implements a model of X doesn’t mean the person can casually reach in and do things with that module.
Edit: After reading your other responses, yeah, “not modeling them as creatures particularly similar to yourself” might just be the correct approach. I, uh, also don’t find people with weak metacognitive skills particularly relatable.
To expand on that...
In my mental ontology, there’s a set of specific concepts and mental motions associated with accountability: viewing people as being responsible for their actions, being disappointed in or impressed by their choices, modeling the assignment of blame/credit as meaningful operations. Implicitly, this requires modeling other people as agents: types of systems which are usefully modeled as having control over their actions. To me, this is a prerequisite for being able to truly connect with someone.
When you apply the not-that-coherent-an-agent lens, you do lose that. Because, like, which parts of that person’s cognition should you interpret as the agent making choices, and which as parts of the malfunctioning exoskeleton the agent has no control over? You can make some decision about that, but this is usually pretty arbitrary. If someone is best modeled like this, they’re not well-modeled as an agent, and holding them accountable is a category error. They’re a type of system that does what it does.
You can still invoke the social rituals of “blame” and “responsibility” if you expect that to change their behavior, but the mental experience of doing so is very different. It’s more like calculating the nudges you need to make to prompt the desired mechanistic behavior, rather than as interfacing with a fellow person. In the latter case, you can sort of relax, communicate in a way focused on transferring information, instead of focusing on the form of communication, and trust them to make correct inferences. In the former case, you need to keep precise track of tone/wording/aesthetics/etc., and it’s less “communication” and more “optimization”.
I really dislike thinking of people in this way, and I try to adopt the viewing-them-as-a-person frame whenever it’s at all possible. But the other frame does unfortunately seem to be useful in many cases. Trying to do otherwise often feels like reaching out for someone’s hand and finding nothing there.
If this is what you meant by viewing others as cats, yeah, that tracks.
Edit: Oh, nice timing.
Yeah, that.