This seems like it’s playing in the same broad hypothesis space as my post on geometric vs lawful general intelligence.
I think your suggestion that norm-enforcement & norm-evasion can use the same structures is probably right, but it had to originate from a seed that’s not purely cat-and-mouse, even if adversarial component subsequently led to quick gains.
The seed seems like it might be something like this: Empathy is useful to mammals in part because it allows kin groups to share basic survival-relevant information. (I don’t fully understand how this works in a way that isn’t magical group selection bullshit, but it seems to happen and we can black-box that confusion for now.) For whatever reason, it’s fitness-enhancing to be able to share information about the sorts of simple action intents that things like body language can communicate. For the same reason, it’s sometimes advantageous to be able to send and receive information with different complex data structures, e.g. sets of conditional instructions. But at some point the most parsimonious way to do that is to have general reasoning skills that can be used to manipulate words as symbols that can be assigned arbitrary referents or functions. Thus—general intelligence!
This seems like it’s playing in the same broad hypothesis space as my post on geometric vs lawful general intelligence.
I think your suggestion that norm-enforcement & norm-evasion can use the same structures is probably right, but it had to originate from a seed that’s not purely cat-and-mouse, even if adversarial component subsequently led to quick gains.
(Cross-posted this comment from Katja’s blog)
The seed seems like it might be something like this: Empathy is useful to mammals in part because it allows kin groups to share basic survival-relevant information. (I don’t fully understand how this works in a way that isn’t magical group selection bullshit, but it seems to happen and we can black-box that confusion for now.) For whatever reason, it’s fitness-enhancing to be able to share information about the sorts of simple action intents that things like body language can communicate. For the same reason, it’s sometimes advantageous to be able to send and receive information with different complex data structures, e.g. sets of conditional instructions. But at some point the most parsimonious way to do that is to have general reasoning skills that can be used to manipulate words as symbols that can be assigned arbitrary referents or functions. Thus—general intelligence!
(Cross-posted this comment from Katja’s blog)