Interesting. Is it fair to say that Mollick’s system is relatively more “serial” with fewer parallelisms at the subcortical level, whereas you’re proposing a system that’s much more “parallel” because there are separate systems doing analogous things at each level? …
Hmm, I guess I’m not really sure what you’re referring to.
Apropos of nothing, is there any role for the visual cortex within your system?
If I recall, V1 isn’t involved in basal ganglia loops, and some higher-level visual areas might project to striatum as “context” but not as part of basal ganglia loops. (I’m not 100% clear on the anatomy here though; I think the literature is confusing to me partly because it took me a while to realize that rat visual cortex is a lot simpler than primate, I’ve heard it’s kinda like “just V1″). So that’s the message of “Is RL Involved in Sensory Processing?”: there’s no RL in the visual cortex AFAICT. Instead I think there’s predictive learning, see for example Randall O’Reilly’s model.
I talk in the main article about “proposal selection”. I think the cortex is just full of little models that make predictions about other little models, and/or predictions about sensory inputs, and/or (self-fulfilling) “predictions” about motor outputs. And if a model is making wrong predictions, it gets thrown out, and over time it gets outright deleted from the system. (The proposals are models too.) So if you’re staring at a dog, you just can’t seriously entertain the proposal “I’m going to milk this cow”. That model involves a prediction that the thing you’re looking at is a cow, and that model in turn is making lower-level predictions about the sensory inputs, and those predictions are being falsified by the actual sensory input, which is a dog not a cow. So the model gets thrown out. It doesn’t matter how high reward you would get for milking a cow, it’s not on the table as a possible proposal.
I believe I noted that the within-cortex proposal-selection / predictive learning algorithms are important things, but declared them out of scope for this particular post.
The last time I wrote anything about the within-cortex algorithm was I guess last year here. These days I’m more excited by the question of “how might we control neocortex-like algorithms?” rather than “how exactly would a neocortex-like algorithm work?”
I too am puzzled about why some people talk about “mPFC” and others talk about “vmPFC”…
Hmm, I guess I’m not really sure what you’re referring to.
If I recall, V1 isn’t involved in basal ganglia loops, and some higher-level visual areas might project to striatum as “context” but not as part of basal ganglia loops. (I’m not 100% clear on the anatomy here though; I think the literature is confusing to me partly because it took me a while to realize that rat visual cortex is a lot simpler than primate, I’ve heard it’s kinda like “just V1″). So that’s the message of “Is RL Involved in Sensory Processing?”: there’s no RL in the visual cortex AFAICT. Instead I think there’s predictive learning, see for example Randall O’Reilly’s model.
I talk in the main article about “proposal selection”. I think the cortex is just full of little models that make predictions about other little models, and/or predictions about sensory inputs, and/or (self-fulfilling) “predictions” about motor outputs. And if a model is making wrong predictions, it gets thrown out, and over time it gets outright deleted from the system. (The proposals are models too.) So if you’re staring at a dog, you just can’t seriously entertain the proposal “I’m going to milk this cow”. That model involves a prediction that the thing you’re looking at is a cow, and that model in turn is making lower-level predictions about the sensory inputs, and those predictions are being falsified by the actual sensory input, which is a dog not a cow. So the model gets thrown out. It doesn’t matter how high reward you would get for milking a cow, it’s not on the table as a possible proposal.
I believe I noted that the within-cortex proposal-selection / predictive learning algorithms are important things, but declared them out of scope for this particular post.
The last time I wrote anything about the within-cortex algorithm was I guess last year here. These days I’m more excited by the question of “how might we control neocortex-like algorithms?” rather than “how exactly would a neocortex-like algorithm work?”
Thanks, that was helpful