Interesting. Is it fair to say that Mollick’s system is relatively more “serial” with fewer parallelisms at the subcortical level, whereas you’re proposing a system that’s much more “parallel” because there are separate systems doing analogous things at each level? I think that parallel arrangement is probably the thing I’ve learned most personally from reading your work. Maybe I just hadn’t thought about it because I focus too much on valuation and PFC decision-making stuff and don’t look broadly enough at movement and other systems.
Apropos of nothing, is there any role for the visual cortex within your system?
I too am puzzled about why some people talk about “mPFC” and others talk about “vmPFC”. I focus on “vmPFC”, mostly because that’s what people in my field talk about. “vmPFC” focuses much more on valuation systems. Theoretically I guess “mPFC” would also include the dorsomedial prefrontal cortex, which includes the anterior cingulate cortex, I guess some systems related to executive control, perhaps response inhibition (although that’s usually quite lateral), perhaps abstract processing. Tends to be a bit of a decision-making homunculous of sorts :/ And then there’s the ACC, whose role in various things is fairly well defined.
So maybe authors who talk about the mPFC aren’t as concerned about distinguishing value processing from all those other things.
Interesting. Is it fair to say that Mollick’s system is relatively more “serial” with fewer parallelisms at the subcortical level, whereas you’re proposing a system that’s much more “parallel” because there are separate systems doing analogous things at each level? …
Hmm, I guess I’m not really sure what you’re referring to.
Apropos of nothing, is there any role for the visual cortex within your system?
If I recall, V1 isn’t involved in basal ganglia loops, and some higher-level visual areas might project to striatum as “context” but not as part of basal ganglia loops. (I’m not 100% clear on the anatomy here though; I think the literature is confusing to me partly because it took me a while to realize that rat visual cortex is a lot simpler than primate, I’ve heard it’s kinda like “just V1″). So that’s the message of “Is RL Involved in Sensory Processing?”: there’s no RL in the visual cortex AFAICT. Instead I think there’s predictive learning, see for example Randall O’Reilly’s model.
I talk in the main article about “proposal selection”. I think the cortex is just full of little models that make predictions about other little models, and/or predictions about sensory inputs, and/or (self-fulfilling) “predictions” about motor outputs. And if a model is making wrong predictions, it gets thrown out, and over time it gets outright deleted from the system. (The proposals are models too.) So if you’re staring at a dog, you just can’t seriously entertain the proposal “I’m going to milk this cow”. That model involves a prediction that the thing you’re looking at is a cow, and that model in turn is making lower-level predictions about the sensory inputs, and those predictions are being falsified by the actual sensory input, which is a dog not a cow. So the model gets thrown out. It doesn’t matter how high reward you would get for milking a cow, it’s not on the table as a possible proposal.
I believe I noted that the within-cortex proposal-selection / predictive learning algorithms are important things, but declared them out of scope for this particular post.
The last time I wrote anything about the within-cortex algorithm was I guess last year here. These days I’m more excited by the question of “how might we control neocortex-like algorithms?” rather than “how exactly would a neocortex-like algorithm work?”
I too am puzzled about why some people talk about “mPFC” and others talk about “vmPFC”…
Interesting. Is it fair to say that Mollick’s system is relatively more “serial” with fewer parallelisms at the subcortical level, whereas you’re proposing a system that’s much more “parallel” because there are separate systems doing analogous things at each level? I think that parallel arrangement is probably the thing I’ve learned most personally from reading your work. Maybe I just hadn’t thought about it because I focus too much on valuation and PFC decision-making stuff and don’t look broadly enough at movement and other systems.
Apropos of nothing, is there any role for the visual cortex within your system?
I too am puzzled about why some people talk about “mPFC” and others talk about “vmPFC”. I focus on “vmPFC”, mostly because that’s what people in my field talk about. “vmPFC” focuses much more on valuation systems. Theoretically I guess “mPFC” would also include the dorsomedial prefrontal cortex, which includes the anterior cingulate cortex, I guess some systems related to executive control, perhaps response inhibition (although that’s usually quite lateral), perhaps abstract processing. Tends to be a bit of a decision-making homunculous of sorts :/ And then there’s the ACC, whose role in various things is fairly well defined.
So maybe authors who talk about the mPFC aren’t as concerned about distinguishing value processing from all those other things.
Hmm, I guess I’m not really sure what you’re referring to.
If I recall, V1 isn’t involved in basal ganglia loops, and some higher-level visual areas might project to striatum as “context” but not as part of basal ganglia loops. (I’m not 100% clear on the anatomy here though; I think the literature is confusing to me partly because it took me a while to realize that rat visual cortex is a lot simpler than primate, I’ve heard it’s kinda like “just V1″). So that’s the message of “Is RL Involved in Sensory Processing?”: there’s no RL in the visual cortex AFAICT. Instead I think there’s predictive learning, see for example Randall O’Reilly’s model.
I talk in the main article about “proposal selection”. I think the cortex is just full of little models that make predictions about other little models, and/or predictions about sensory inputs, and/or (self-fulfilling) “predictions” about motor outputs. And if a model is making wrong predictions, it gets thrown out, and over time it gets outright deleted from the system. (The proposals are models too.) So if you’re staring at a dog, you just can’t seriously entertain the proposal “I’m going to milk this cow”. That model involves a prediction that the thing you’re looking at is a cow, and that model in turn is making lower-level predictions about the sensory inputs, and those predictions are being falsified by the actual sensory input, which is a dog not a cow. So the model gets thrown out. It doesn’t matter how high reward you would get for milking a cow, it’s not on the table as a possible proposal.
I believe I noted that the within-cortex proposal-selection / predictive learning algorithms are important things, but declared them out of scope for this particular post.
The last time I wrote anything about the within-cortex algorithm was I guess last year here. These days I’m more excited by the question of “how might we control neocortex-like algorithms?” rather than “how exactly would a neocortex-like algorithm work?”
Thanks, that was helpful