I understand Ivan’s first point. My main concern is that we don’t have the right processes laid out for these models to follow. In the end, we want these models to determine their own process of doing things (if we don’t find a way to emulate human brain processes into machines) and establishing a clear-cut process for tasks could limit the model’s creativity. We would have to have a perfect model of how each of these NN tasks should be carried out.
However, the idea of combining the two is interested. As research suggests, backprop and a global update function doesn’t exist in the brain (although large sections of the brain can carry out orchestrated tasks amazingly). There must be a combination of local updates to these synaptic weights (aligned with specific process-based tasks) which follow some global loss function in the brain. It’d be interesting to get more thoughts on this.
I understand Ivan’s first point. My main concern is that we don’t have the right processes laid out for these models to follow. In the end, we want these models to determine their own process of doing things (if we don’t find a way to emulate human brain processes into machines) and establishing a clear-cut process for tasks could limit the model’s creativity. We would have to have a perfect model of how each of these NN tasks should be carried out.
However, the idea of combining the two is interested. As research suggests, backprop and a global update function doesn’t exist in the brain (although large sections of the brain can carry out orchestrated tasks amazingly). There must be a combination of local updates to these synaptic weights (aligned with specific process-based tasks) which follow some global loss function in the brain. It’d be interesting to get more thoughts on this.