That’s what I imagine the simplest embedded agents look like: info in, finite optimizer circuit, one single decision out, whole thing is a finite chunk of circuitry.
I really haven’t thought very hard about this subject, so pardon the confused comment.
I feel like that’s a type of embedded agent, but it’s not much like my actual experience of embedded agents (nor a simplified version of it). Like, there’s many much more granular levels of information processing between me and the environment. Do I count as my knee reflex that kicks out? Do I count as the part of me that responds very suddenly and almost reflexively to pain (though I can override those impulses)? Sometimes I build pieces of code or art or essays into the environment that feel like extensions of myself. Sometimes I repeatedly do things that no part of me endorses like picking scabs (for others: smoking).
I mention all of these to point to me not being sure which part of me to actually draw the boundary around as “the agent”. There are lots of adaptation-executions which are more intertwined with the environment than with the optimising part of me, and sometimes I identify more with parts of the environment I built than with those adaptations I sometimes execute—those parts of the environment are continuing to optimise for something I care about more than some parts of my nervous system.
Added: It sounds to me like you’re modelling the simple case as one with a particular clear dividing line between decision-making-parts and rest-of-environment, whereas I don’t know why you get to assume that particular line, and it doesn’t seem much like a simplified version of me. I don’t expect there is a fact of the matter about which part of this world is ‘me optimising’ and which parts aren’t, but that I have to somehow reduce ‘me’ or something to have a more granular model of the world. Like, my bedroom optimises for certain aesthetic experiences and affordances for its inhabitants, like encouraging them to read more and get enough fresh air, and this feels more like ‘me optimising’ than the part of me that’s startled by loud noises.
Not sure if this is the same thing you’re pointing at, but there’s a cybernetics/predictive processing view that pictures humans (and other agenty things) as being made up of a bunch of feedback control systems layered on top of each other. I imagine a theory of embedded agency which would be able to talk about each of those little feedback controls as an “agent” in itself: it takes in data, chews on it, and outputs decisions to achieve some goal.
Another piece which may relate to what you’re pointing at: I expect the “boundary” of an agent to be fuzzy on the “inputs” side, and less fuzzy but still flexible on the “outputs” side. On the inputs side, there’s a whole chain of cause-and-effect which feeds data into my brain, and there’s some freedom in whether to consider “me” to begin at e.g. the eye, or the photoreceptor, or the optic nerve, or… On the outputs side, there’s a clearer criterion for what’s “me”: it’s whatever things I’m “choosing” when I optimize, i.e. anything I assume I control for planning purposes. That’s a sharper criterion, but it still leaves a lot of flexibility—e.g. I can consider my car a part of “me” while I’m driving it. Point is, when I say “draw a box”, I do imagine having some freedom in where the boundary goes—the boundary is just there to help point out roughly which part of the universe we’re talking about.
I really haven’t thought very hard about this subject, so pardon the confused comment.
I feel like that’s a type of embedded agent, but it’s not much like my actual experience of embedded agents (nor a simplified version of it). Like, there’s many much more granular levels of information processing between me and the environment. Do I count as my knee reflex that kicks out? Do I count as the part of me that responds very suddenly and almost reflexively to pain (though I can override those impulses)? Sometimes I build pieces of code or art or essays into the environment that feel like extensions of myself. Sometimes I repeatedly do things that no part of me endorses like picking scabs (for others: smoking).
I mention all of these to point to me not being sure which part of me to actually draw the boundary around as “the agent”. There are lots of adaptation-executions which are more intertwined with the environment than with the optimising part of me, and sometimes I identify more with parts of the environment I built than with those adaptations I sometimes execute—those parts of the environment are continuing to optimise for something I care about more than some parts of my nervous system.
Added: It sounds to me like you’re modelling the simple case as one with a particular clear dividing line between decision-making-parts and rest-of-environment, whereas I don’t know why you get to assume that particular line, and it doesn’t seem much like a simplified version of me. I don’t expect there is a fact of the matter about which part of this world is ‘me optimising’ and which parts aren’t, but that I have to somehow reduce ‘me’ or something to have a more granular model of the world. Like, my bedroom optimises for certain aesthetic experiences and affordances for its inhabitants, like encouraging them to read more and get enough fresh air, and this feels more like ‘me optimising’ than the part of me that’s startled by loud noises.
Not sure if this is the same thing you’re pointing at, but there’s a cybernetics/predictive processing view that pictures humans (and other agenty things) as being made up of a bunch of feedback control systems layered on top of each other. I imagine a theory of embedded agency which would be able to talk about each of those little feedback controls as an “agent” in itself: it takes in data, chews on it, and outputs decisions to achieve some goal.
Another piece which may relate to what you’re pointing at: I expect the “boundary” of an agent to be fuzzy on the “inputs” side, and less fuzzy but still flexible on the “outputs” side. On the inputs side, there’s a whole chain of cause-and-effect which feeds data into my brain, and there’s some freedom in whether to consider “me” to begin at e.g. the eye, or the photoreceptor, or the optic nerve, or… On the outputs side, there’s a clearer criterion for what’s “me”: it’s whatever things I’m “choosing” when I optimize, i.e. anything I assume I control for planning purposes. That’s a sharper criterion, but it still leaves a lot of flexibility—e.g. I can consider my car a part of “me” while I’m driving it. Point is, when I say “draw a box”, I do imagine having some freedom in where the boundary goes—the boundary is just there to help point out roughly which part of the universe we’re talking about.