To us, this whole train of thought[2] has further reinforced the connection that modularity = abstraction, in some sense. Whenever you give an optimiser perfect information on its task in training, you get a fine-tuned mess of a network humans find hard to understand. But if you want something that can handle unforeseen, yet in some sense natural changes, something modular performs better.
I remember a deep mind paper where a NN was trained on a variety of environments to be able to handle (not known before hand real world parameters in a robot hand/the world (friction etc.)). Did that result in, or involve, a modular network?
It seems like an environment that changes might cause modularity. Though, aside from trying to make something modular, it seem like it could potentially fall out of stuff like ‘we want something that’s easier to train’. Though I didn’t see an explanation in this post of why deep learning would do this (though it was mentioned that they do and evolutionary algorithms wouldn’t, EA not resulting in modularity seemed like it was explored, while deep learning wasn’t).
Do these architectures somehow enforce something equivalent to a connection cost?
In other words, I’m wondering, does a connection cost help with training?
It seems like an environment that changes might cause modularity. Though, aside from trying to make something modular, it seem like it could potentially fall out of stuff like ‘we want something that’s easier to train’.
This seems really interesting in the biological context, and not something we discussed much in the other post. For instance, if you had two organisms, one modular and one not modular, even if there’s currently no selection advantage for the modular one, it might just be trained much faster and hence be more likely to hit on a good solution before the nonmodular network (i.e. just because it’s searching over parameter space at a larger rate).
it might just be trained much faster and hence be more likely to hit on a good solution before the nonmodular network (i.e. just because it’s searching over parameter space at a larger rate).
Or the less modular one can’t train (evolve) as fast when the environment changes. (Or, it changes faster enabling it to travel to different environments.)
Biology kind of does both (modular and integrated), a lot. Like, I want to say part of why the brain is hard to understand is because of how integrated it is. What’s going on in the brain? I saw one answer to this that says ‘it is this complicated in order to obfuscate, to make it harder to hack, this mess has been shaped by parasites, which it is designed to shake off, that is why it is a mess, and might just throw some neurotoxin in there. Why? To kill stiff that’s trying to mess around in there.’ (That is just from memory/reading a reviews on a blog, and you should read the paper/later work https://www.journals.uchicago.edu/doi/10.1086/705038)
I want to say integrated a) (often) isn’t as good (separating concerns is better), but b) it’s cheaper to re-use stuff, and have it solve multiple purposes. Breathing through the same area you drink water/eat food through can cause issues. But integrating also allows improvements/increased efficiency (although I want to say, in systems people make, it can make it harder to refine or improve the design).
I recall something similar about a robot hand trained in varying simulations. I remember an OpenAI project not a Deepmind one… Here’s the link to the OpenAI environment-varying learner: https://openai.com/blog/learning-dexterity/
I remember a deep mind paper where a NN was trained on a variety of environments to be able to handle (not known before hand real world parameters in a robot hand/the world (friction etc.)). Did that result in, or involve, a modular network?
It seems like an environment that changes might cause modularity. Though, aside from trying to make something modular, it seem like it could potentially fall out of stuff like ‘we want something that’s easier to train’. Though I didn’t see an explanation in this post of why deep learning would do this (though it was mentioned that they do and evolutionary algorithms wouldn’t, EA not resulting in modularity seemed like it was explored, while deep learning wasn’t).
In other words, I’m wondering, does a connection cost help with training?
This seems really interesting in the biological context, and not something we discussed much in the other post. For instance, if you had two organisms, one modular and one not modular, even if there’s currently no selection advantage for the modular one, it might just be trained much faster and hence be more likely to hit on a good solution before the nonmodular network (i.e. just because it’s searching over parameter space at a larger rate).
Or the less modular one can’t train (evolve) as fast when the environment changes. (Or, it changes faster enabling it to travel to different environments.)
Biology kind of does both (modular and integrated), a lot. Like, I want to say part of why the brain is hard to understand is because of how integrated it is. What’s going on in the brain? I saw one answer to this that says ‘it is this complicated in order to obfuscate, to make it harder to hack, this mess has been shaped by parasites, which it is designed to shake off, that is why it is a mess, and might just throw some neurotoxin in there. Why? To kill stiff that’s trying to mess around in there.’ (That is just from memory/reading a reviews on a blog, and you should read the paper/later work https://www.journals.uchicago.edu/doi/10.1086/705038)
I want to say integrated a) (often) isn’t as good (separating concerns is better), but b) it’s cheaper to re-use stuff, and have it solve multiple purposes. Breathing through the same area you drink water/eat food through can cause issues. But integrating also allows improvements/increased efficiency (although I want to say, in systems people make, it can make it harder to refine or improve the design).
I recall something similar about a robot hand trained in varying simulations. I remember an OpenAI project not a Deepmind one… Here’s the link to the OpenAI environment-varying learner: https://openai.com/blog/learning-dexterity/
I mixed up deepmind and openai.