I’ve been using “natural abstraction” here as if it just means an abstraction that would be useful for a wide variety of agents to have in their toolbox. But we might also use “natural abstractions” to denote the vital abstractions, those that aren’t merely nice to have, but that you literally can’t complete certain tasks without using.
The thing I usually have in mind these days is stronger than the first but weaker than the second. Roughly speaking: natural abstractions should be convergent for distributed system produced by local selection pressures. That’s a stronger condition than “useful to have in toolbox” (since not all useful structures are accessible to local search), but weaker than “literally can’t complete certain tasks without it”.
The thing I usually have in mind these days is stronger than the first but weaker than the second. Roughly speaking: natural abstractions should be convergent for distributed system produced by local selection pressures. That’s a stronger condition than “useful to have in toolbox” (since not all useful structures are accessible to local search), but weaker than “literally can’t complete certain tasks without it”.