This question also has a negative answer, as witnessed by the example of an ant colony—agent-like behavior without agent-like architecture, produced by a “non-agenty” optimization process of evolution. Nonetheless, a general version of the question remains: If some X exhibits agent-like behavior, does it follow that there exists some interesting physical structure causally upstream of X?
Neat example! But for my part, I’m confused about this last sentence, even after reading the footnote:
An example of such “interesting physical structure” would be an implementation of an optimization architecture.
For one thing, I’m not sure I have much intuition about what is meant by “optimization architecture”. For instance, I would not know how to begin answering the question:
Does optimization behavior imply optimization architecture?
And I have even less of a clue what is intended by “interesting physical structure” (perhaps facetiously, any process that causes agent-like behavior to arise sounds “interesting” for that reason alone).
In your ant colony example, is evolution the “interesting physical structure”, and if so, how is it a physical structure?
First off, while I feel somewhat de-confused about X-like behavior, I don’t feel very confident about X-like architectures. Maybe the meaning is somewhat clear on higher levels of abstraction (e.g., if my brain goes “realize I want to describe a concept --> visualize several explanations and judge each for suitability --> pick the one that seems the best --> send a signal to start typing it down”, then this would be a kind of search/optimization-thingy). But on the level of physics, I don’t really know what an architecture means. So take this with a grain of salt.
Maybe the term “physical structure” is misleading. The thing I was trying to point at is the distinction between being able to accurately model Y using model X, and Y actually being X. In the sense that there might be a giant look-up table (GLUT) that accuractly predicts your behavior, but on no level of abstraction is it correct to say that you actually are a GLUT. Whereas modelling you as having some goals, planning, etc. might be less accurate but somewhat more, hm, true. I realize this isn’t very precise, but I guess you can see what I mean.
That being said, I suppose that what I meant by “optimization architecture” is, for example, a stochastic gradient descent with the emphasis on “this is the input”, “this is the part of the algorithm that does the calculation”, and “this is the output”. An “implementation of an optimization architecture” would be...well, the atoms of your computer that perform SGD, or maybe some simple bacteria that moves in the direction where the concentration of whatever-it-likes is the highest (not that anything I know would implement precisely SGD, but still).
Ad “interesting physical structure” behind the ant-colony: If by “evolution” we mean the atoms that the world is made of, as they changed over time until your ant colony emerged...then yeah, this is a physical structure causally upstream of the ant colony, and one that is responsible for the ant colony behaving the way it does.
I wouldn’t say it is interesting (to me, and w.r.t. the ant colony) though, since it is totally incomprehensible to me. (But maybe “interestingness” doesn’t really make sense on the level of physics, and is only relevant in relation to our abstract world-models and their understanding.)
Finally, the ideal thing a “X-like behavior ==> Y-like architecture” theorem would cash out into is a criterion that you can actually check and say with certainty that the thing will not exhibit X-like behavior. (Whether this is reasonable to hope for is another matter.) So, even if all that I have written in this comment turns out to be nonsense, getting such criterion is what we are after :-).
Neat example! But for my part, I’m confused about this last sentence, even after reading the footnote:
For one thing, I’m not sure I have much intuition about what is meant by “optimization architecture”. For instance, I would not know how to begin answering the question:
Does optimization behavior imply optimization architecture?
And I have even less of a clue what is intended by “interesting physical structure” (perhaps facetiously, any process that causes agent-like behavior to arise sounds “interesting” for that reason alone).
In your ant colony example, is evolution the “interesting physical structure”, and if so, how is it a physical structure?
First off, while I feel somewhat de-confused about X-like behavior, I don’t feel very confident about X-like architectures. Maybe the meaning is somewhat clear on higher levels of abstraction (e.g., if my brain goes “realize I want to describe a concept --> visualize several explanations and judge each for suitability --> pick the one that seems the best --> send a signal to start typing it down”, then this would be a kind of search/optimization-thingy). But on the level of physics, I don’t really know what an architecture means. So take this with a grain of salt.
Maybe the term “physical structure” is misleading. The thing I was trying to point at is the distinction between being able to accurately model Y using model X, and Y actually being X. In the sense that there might be a giant look-up table (GLUT) that accuractly predicts your behavior, but on no level of abstraction is it correct to say that you actually are a GLUT. Whereas modelling you as having some goals, planning, etc. might be less accurate but somewhat more, hm, true. I realize this isn’t very precise, but I guess you can see what I mean.
That being said, I suppose that what I meant by “optimization architecture” is, for example, a stochastic gradient descent with the emphasis on “this is the input”, “this is the part of the algorithm that does the calculation”, and “this is the output”. An “implementation of an optimization architecture” would be...well, the atoms of your computer that perform SGD, or maybe some simple bacteria that moves in the direction where the concentration of whatever-it-likes is the highest (not that anything I know would implement precisely SGD, but still).
Ad “interesting physical structure” behind the ant-colony: If by “evolution” we mean the atoms that the world is made of, as they changed over time until your ant colony emerged...then yeah, this is a physical structure causally upstream of the ant colony, and one that is responsible for the ant colony behaving the way it does. I wouldn’t say it is interesting (to me, and w.r.t. the ant colony) though, since it is totally incomprehensible to me. (But maybe “interestingness” doesn’t really make sense on the level of physics, and is only relevant in relation to our abstract world-models and their understanding.)
Finally, the ideal thing a “X-like behavior ==> Y-like architecture” theorem would cash out into is a criterion that you can actually check and say with certainty that the thing will not exhibit X-like behavior. (Whether this is reasonable to hope for is another matter.) So, even if all that I have written in this comment turns out to be nonsense, getting such criterion is what we are after :-).