This is a great point, and very nicely made—but I do think it avoids the topic of why people end up in these styles of argument in the first place.
I think there would be more value in discussing How to deal with Mob & Bailey situations once they arise rather than How to stop Mob & Bailey situations from arising.
You point out, correctly, that Mob & Bailey situations tend to occur when one is overly anthropomorphising a group of people, as though that group were an individual person.
The real problem is, that there are situations where it really is useful to act that way for pragmatic reasons.
At least in my experience, Mob & Bailey arguments tend to happen in fairly broad discussions about group behaviours—perhaps about ideologies, or social practices between groups.
These are situations where it is very useful and important to be able to try and address an entire group’s collective behaviour, moreso than addressing how individuals act and rationalise their decisions.
In these cases, what we’re really trying to discuss is the mechanics of how groups are organised, rather than any one individual’s beliefs.
If we’re arguing with a Tautology Club which breaks university rules, then talking with the club president does make sense. If we’re arguing with a military, then talking with the Secretary of Defense isn’t a bad place to start—but perhaps investigating middle-management positions would be more practical.
I’d love to read further ideas more along the lines of “How to deal with Mob & Bailey situations”, because I think the results can be quite different depending on which structure of social group you’re arguing with.
Interesting idea, but it seems risky.
Would life be the only, or for that matter, even the primary, complex system that such an AI would avoid interfering with?
Further, it seems likely that a curiosity-based AI might intentionally create or seek out complexity, which could be risky.
Think of how kids love to say “I want to go to the moon!” “I want to go to every country in the world!”. I mean, I do too and I’m an adult. Surely a curiosity-based AI would attempt to go to fairly extreme limits for the sake of satiating its own curiosity, at the expense of other values.
Maybe such an AGI could have like… an allowance? “Never spend more than 1% of your resources on a single project” or something? But I have absolutely no idea how you could define a consistent idea of a “single project”.