This is so great, I find that this so satisfyingly ties together a bunch of piecemeal understandings in my head.
Maybe it’s not worth getting into, because it’s more about understanding humans than the general case of shard-based agents, but… Human brains have a lot of weird bugs that can lead to accidental shard creations / shifts and other stuff like optical illusions or certain drugs being more addictive than would be predicted based on the amount of subjective pleasure they seem to deliver based on idiosyncrasies of how they activate the reward systems.
Or like how the local plasticity of the cortex, which allows modules to learn, and also allows for local learning to reallocate module territory on the borders from one module to the other, can lead sometimes to information leaks between modules that can end up accidentally reinforced. Like sensory leaks between skin areas which aren’t physically co-located on the body but whose receptive fields in the brain are next to each other and compete for territory. That’s an example of something I wouldn’t attempt to reproduce if I were trying to make a shard-based / brain-like agent.
This is so great, I find that this so satisfyingly ties together a bunch of piecemeal understandings in my head. Maybe it’s not worth getting into, because it’s more about understanding humans than the general case of shard-based agents, but… Human brains have a lot of weird bugs that can lead to accidental shard creations / shifts and other stuff like optical illusions or certain drugs being more addictive than would be predicted based on the amount of subjective pleasure they seem to deliver based on idiosyncrasies of how they activate the reward systems. Or like how the local plasticity of the cortex, which allows modules to learn, and also allows for local learning to reallocate module territory on the borders from one module to the other, can lead sometimes to information leaks between modules that can end up accidentally reinforced. Like sensory leaks between skin areas which aren’t physically co-located on the body but whose receptive fields in the brain are next to each other and compete for territory. That’s an example of something I wouldn’t attempt to reproduce if I were trying to make a shard-based / brain-like agent.