The principle here is competition among populations of neurons. The purpose is to reduce crosstalk. Higher brain regions can focus on processing only the stuff you’re attending to because most of their inputs have been down-regulated so only the attended ones are sending information.
The principal operates by simple competition. If I’m thinking about colors, higher areas are representing colors. That activates lower areas/neurons representing colors. because they’re wired together by associative learning (or just about any useful learning rule will connect semantically related representations).
There are probably some particular flourishes evolution used to amplify the efficiency (like competition at the level of thalamic reticular nucleus that then regulates whole lower cortical regions, and synchronous firing of attended/active neurons to further sharpen their win over unattended neural populations), but the central principal is indeed very neat.
I did my Master’s directly on this, PhD thesis on competition/attention for the purposes of visual search, and have kept it top of mind as a central principal of brain function.
Attention in transformers is different but has the same broad outlines in function.
The principle here is competition among populations of neurons. The purpose is to reduce crosstalk. Higher brain regions can focus on processing only the stuff you’re attending to because most of their inputs have been down-regulated so only the attended ones are sending information.
The principal operates by simple competition. If I’m thinking about colors, higher areas are representing colors. That activates lower areas/neurons representing colors. because they’re wired together by associative learning (or just about any useful learning rule will connect semantically related representations).
There are probably some particular flourishes evolution used to amplify the efficiency (like competition at the level of thalamic reticular nucleus that then regulates whole lower cortical regions, and synchronous firing of attended/active neurons to further sharpen their win over unattended neural populations), but the central principal is indeed very neat.
I did my Master’s directly on this, PhD thesis on competition/attention for the purposes of visual search, and have kept it top of mind as a central principal of brain function.
Attention in transformers is different but has the same broad outlines in function.