What do you mean by “an algorithm that we think of as happening in one step”?
I think of it as analysis-by-synthesis, a.k.a. “search through a space of generative models for one that matches the data”. The search process doesn’t have to be fast, let alone one step—the splotchy pictures in this post are a good example, you stare at them for a while until they snap into place. Right? Or sorry if I’m misunderstanding your point.
I’m saying the abstraction of (e.g.) CNNs as doing their forward pass all in one timestep does not apply to the brain. So I think we agree and I just wasn’t too clear.
For CNNs we don’t worry about top-down control intervening in the middle of a forward pass, and to the extent that engineers might increase chip efficiency by having different operations be done simultaneously, we usually want to ensure that they can’t interfere with each other, maintaining the layer of abstraction. But the human visual cortex probably violates these assumptions not just out of necessity, but gains advantages.
What do you mean by “an algorithm that we think of as happening in one step”?
I think of it as analysis-by-synthesis, a.k.a. “search through a space of generative models for one that matches the data”. The search process doesn’t have to be fast, let alone one step—the splotchy pictures in this post are a good example, you stare at them for a while until they snap into place. Right? Or sorry if I’m misunderstanding your point.
I’m saying the abstraction of (e.g.) CNNs as doing their forward pass all in one timestep does not apply to the brain. So I think we agree and I just wasn’t too clear.
For CNNs we don’t worry about top-down control intervening in the middle of a forward pass, and to the extent that engineers might increase chip efficiency by having different operations be done simultaneously, we usually want to ensure that they can’t interfere with each other, maintaining the layer of abstraction. But the human visual cortex probably violates these assumptions not just out of necessity, but gains advantages.