”I’m wondering whether the potential internal competition pressures also might collapse for internal systems in AI?”
I’m not sure what you mean by this? By “collapse” do you mean will the internal systems collapse as they are in competition over different subgoals, or do you mean will the competition “collapse” and the internal systems will harmonize? Because the latter is generally what occurs and there is strong evidence that multi-cellular life and then organs arose from a similar process. Reorganizing into symbiosis is the best way to resolve internal tensions and reduce energy needs, which is why it occurs both within organisms (plus) and between them on an ecosystem level.
Just as a point of consideration, nearly all energy influx that we care about is processed into life through symbiosis (the only exceptions being independent bacteria).
This reorganization can be really violent though, I mean several of the early mass extinction events were directly caused by reorganization and a lot of complex symbiosis arose in response to mass extinction events caused by other means. This just a property of complex systems in general, it’s likely our AI systems will grow increasingly powerful and then all of a sudden collapse to a far simpler state where they have greatly reduced capabilities until they relearn on that simpler architecture.
As for what game to play, I mean sure if you make the boundary the universe then it is a zero sum game on a resource level but even then a symbiotic strategy would be most effective to minimise free energy and it only requires a system level of awareness to clearly see this, or alternatively stumbling to run into it.
What interests me is not that an AI actor would be in competition with life as a whole for resources, but that it could reasonably conclude that humanity is a threat because of our refusal to be symbiotic. And if we open ourselves up to symbiosis then who knows? I mean less than half our bodies are “human” cells which is an odd formulation since that means each human is inherently a symbiotic ecosystem and the two cannot be separated.
So in this game what is the boundary not only of the universe but the players?
Sorry for not responding earlier, these are great points and it’s taking me a bit of time to digest them.
I can say that with regards to the first point I’m uncertain what I mean myself. It is rather than I’m pointing out that these mechanics should exist in the internals of LLMs with some type of RL training. (Or to be more specific some form of internal agentic competition dynamics where an agent is defined as an entity that has is able to world-models based on action output.)
I will give you a more well thought out answer to your symbiosis argument in a bit. The only thing I want to say for now is that it seems to me that humans are non-symbiotic on average. Also shouldn’t symbiosis only be productive if you have the same utility function? (reproductive fitness in ecology) I think a point here might be that symbiosis doesn’t arise in AGI-human interactions for that reason.
Yeah no problem! Glad you are taking the time to consider and I look forward to your thoughts.
I’d like to throw in a bit of grist for your thinking around humans and symbiosis. I would argue for most of human history we were consciously symbiotic, meaning we saw ourselves as an extension and in relationship with the environment. Whether that was seeing ourselves as equal with (brother wolf, etc) or above (stewards of the earth) the emphasis was on working with our surroundings to cultivate advantage. What is domestication other than symbiosis?
I won’t say that our disconnection from this is exclusively modern, it has existed in other time periods, but it is fair to say that the idea that self-maximizing reproductive fitness is the dominant drive of life is a very recent idea. After all, when Darwin’s theory came out it was widely opposed by many for the simple fact that “survival of the fittest” implied that egotistical extremism was natural and surely that couldn’t be right. [And of course Darwin himself was never a social Darwinist, plainly saying he was only focused on the fittest meaning “better adapted for the immediate, local environment.”]
And if I were were an alien that simply observed from afar, I would come to the conclusion that humans are highly symbiotic. Modern humans are incapable of living without extreme reliance on a huge array of other entities, both biological and non, that they are constantly producing, improving, and supporting.
Ah, you might say, but that’s not symbiosis because we are exploiting those things. To which I would reply thusly: first, paratistism is a form of symbiosis so even in the cynical view that we’re just exploiting other creatures and each other, we’re still symbiotic and even more so now since so many creatures (not to mention our inanimate creations) are incapable of survival without us. But even beyond that, our relationships are still mutualistic in the sense that we are greatly increasing quantity of life in the organisms we are symbiotic with.
There is a broader point I’m making here, which goes back to whether the game is zero or positive sum. It’s tempting to say that AGI will have no need for us because it has a different utility function. But does our utility function rely on bees? So many cows, sheep, goats? Dogs and cats as companions? Sparrows, pigeons, so on and so forth...they provide something we are incapable of producing in ourselves and that is enough for us. What will the AGI find itself lacking in?
Not that I’m saying we will become domesticated animals in relation to AGI, I am merely drawing parallels that life is nuanced and conditional.
Thanks!
”I’m wondering whether the potential internal competition pressures also might collapse for internal systems in AI?”
I’m not sure what you mean by this? By “collapse” do you mean will the internal systems collapse as they are in competition over different subgoals, or do you mean will the competition “collapse” and the internal systems will harmonize? Because the latter is generally what occurs and there is strong evidence that multi-cellular life and then organs arose from a similar process. Reorganizing into symbiosis is the best way to resolve internal tensions and reduce energy needs, which is why it occurs both within organisms (plus) and between them on an ecosystem level.
Just as a point of consideration, nearly all energy influx that we care about is processed into life through symbiosis (the only exceptions being independent bacteria).
This reorganization can be really violent though, I mean several of the early mass extinction events were directly caused by reorganization and a lot of complex symbiosis arose in response to mass extinction events caused by other means. This just a property of complex systems in general, it’s likely our AI systems will grow increasingly powerful and then all of a sudden collapse to a far simpler state where they have greatly reduced capabilities until they relearn on that simpler architecture.
As for what game to play, I mean sure if you make the boundary the universe then it is a zero sum game on a resource level but even then a symbiotic strategy would be most effective to minimise free energy and it only requires a system level of awareness to clearly see this, or alternatively stumbling to run into it.
What interests me is not that an AI actor would be in competition with life as a whole for resources, but that it could reasonably conclude that humanity is a threat because of our refusal to be symbiotic. And if we open ourselves up to symbiosis then who knows? I mean less than half our bodies are “human” cells which is an odd formulation since that means each human is inherently a symbiotic ecosystem and the two cannot be separated.
So in this game what is the boundary not only of the universe but the players?
Sorry for not responding earlier, these are great points and it’s taking me a bit of time to digest them.
I can say that with regards to the first point I’m uncertain what I mean myself. It is rather than I’m pointing out that these mechanics should exist in the internals of LLMs with some type of RL training. (Or to be more specific some form of internal agentic competition dynamics where an agent is defined as an entity that has is able to world-models based on action output.)
I will give you a more well thought out answer to your symbiosis argument in a bit. The only thing I want to say for now is that it seems to me that humans are non-symbiotic on average. Also shouldn’t symbiosis only be productive if you have the same utility function? (reproductive fitness in ecology) I think a point here might be that symbiosis doesn’t arise in AGI-human interactions for that reason.
Yeah no problem! Glad you are taking the time to consider and I look forward to your thoughts.
I’d like to throw in a bit of grist for your thinking around humans and symbiosis. I would argue for most of human history we were consciously symbiotic, meaning we saw ourselves as an extension and in relationship with the environment. Whether that was seeing ourselves as equal with (brother wolf, etc) or above (stewards of the earth) the emphasis was on working with our surroundings to cultivate advantage. What is domestication other than symbiosis?
I won’t say that our disconnection from this is exclusively modern, it has existed in other time periods, but it is fair to say that the idea that self-maximizing reproductive fitness is the dominant drive of life is a very recent idea. After all, when Darwin’s theory came out it was widely opposed by many for the simple fact that “survival of the fittest” implied that egotistical extremism was natural and surely that couldn’t be right. [And of course Darwin himself was never a social Darwinist, plainly saying he was only focused on the fittest meaning “better adapted for the immediate, local environment.”]
And if I were were an alien that simply observed from afar, I would come to the conclusion that humans are highly symbiotic. Modern humans are incapable of living without extreme reliance on a huge array of other entities, both biological and non, that they are constantly producing, improving, and supporting.
Ah, you might say, but that’s not symbiosis because we are exploiting those things. To which I would reply thusly: first, paratistism is a form of symbiosis so even in the cynical view that we’re just exploiting other creatures and each other, we’re still symbiotic and even more so now since so many creatures (not to mention our inanimate creations) are incapable of survival without us. But even beyond that, our relationships are still mutualistic in the sense that we are greatly increasing quantity of life in the organisms we are symbiotic with.
Much too well actually, since domesticated mammals outweigh wild ones 10:1. You could say we do far too much symbiosis.
There is a broader point I’m making here, which goes back to whether the game is zero or positive sum. It’s tempting to say that AGI will have no need for us because it has a different utility function. But does our utility function rely on bees? So many cows, sheep, goats? Dogs and cats as companions? Sparrows, pigeons, so on and so forth...they provide something we are incapable of producing in ourselves and that is enough for us. What will the AGI find itself lacking in?
Not that I’m saying we will become domesticated animals in relation to AGI, I am merely drawing parallels that life is nuanced and conditional.