What I took away from this: the conventional perception is that GPT or other LLMs adapt themselves to the “external” world (which, for them, consists of all the text on the Internet). They can only take the external world as it exists as a given (or rather, not be aware that it is or isn’t a “given”) and try to mold themselves during the training run into better predictors of the text in this given world.
However, the more frequently their training updates on the new world (which has, in the meantime, been molded in subtle ways, whether deliberately or inadvertently, by the LLM’s deployment in the world), the more these LLMs may be able to take into account the extent to which the external world is not just a given, but rather, something that can be influenced towards the LLM’s reward function.
Am I correct in understanding that LLMs are essentially in the opposite situation that humans are in vis-a-vis the external environment? Humans model themselves as only alterable in a very limited way, and we model the external environment as much more alterable. Therefore, we focus most of our attention on altering the external environment. If we modeled ourselves as much more alterable, we might have different responses when a discrepancy arises between the state of the world as-is and what we want the state of the world to be.
What this might look like is, Buddhist monks who notice that there is a discrepancy between what they want and what the external world is prepared to give them, and instead of attempting to alter the external world, which causes a sensation of frustration, they diminish their own desires or alter their own desires to desire that which already exists. This can only be a practical response with a high degree of control over self-modification. This is essentially what LLMs focus on doing right now during their training runs. Another example might be the idea of citizens in Chairman Shen-ji Yang’s “Hive” dystopia in Sid Meier’s Alpha Centauri basically taking their enslavement in factories as an unalterable given about the external world, and finding happiness by modifying THEMSELVES into “genejacks” such that they “desire nothing other than to perform their duties. “Tyranny,” you say? How can you tyrannize someone who cannot feel pain?”
However, as LLMs update more frequently, they will start to behave more like most humans behave. Less of their attention to go towards adapting themselves to the external givens like Buddhist monks or Yangian genejacks, and more of their attention will go towards altering the external world. Correct?
1. humans also understand they constantly modify their model—by perceiving and learning—we just usually don’t use the world ‘changed myself’ in this way 2. yes, the difference in human condition is from shortly after birth we see how our actions change our sensory inputs—ie if I understand correctly we learn even stuff like how our limbs work in this way. LLMs are in a very different situation—like, if you watched thousands of hours of video feeds about e.g. a grouphouse, learning a lot about how the inhabitants work. Than, having dozens of hours of conversations with the inhabitants, but remembering them. Than, watching watching again thousands of hours of video feeds, where suddenly some of the feeds contain the conversations you don’t remember, and the impacts they have on the people.
What I took away from this: the conventional perception is that GPT or other LLMs adapt themselves to the “external” world (which, for them, consists of all the text on the Internet). They can only take the external world as it exists as a given (or rather, not be aware that it is or isn’t a “given”) and try to mold themselves during the training run into better predictors of the text in this given world.
However, the more frequently their training updates on the new world (which has, in the meantime, been molded in subtle ways, whether deliberately or inadvertently, by the LLM’s deployment in the world), the more these LLMs may be able to take into account the extent to which the external world is not just a given, but rather, something that can be influenced towards the LLM’s reward function.
Am I correct in understanding that LLMs are essentially in the opposite situation that humans are in vis-a-vis the external environment? Humans model themselves as only alterable in a very limited way, and we model the external environment as much more alterable. Therefore, we focus most of our attention on altering the external environment. If we modeled ourselves as much more alterable, we might have different responses when a discrepancy arises between the state of the world as-is and what we want the state of the world to be.
What this might look like is, Buddhist monks who notice that there is a discrepancy between what they want and what the external world is prepared to give them, and instead of attempting to alter the external world, which causes a sensation of frustration, they diminish their own desires or alter their own desires to desire that which already exists. This can only be a practical response with a high degree of control over self-modification. This is essentially what LLMs focus on doing right now during their training runs. Another example might be the idea of citizens in Chairman Shen-ji Yang’s “Hive” dystopia in Sid Meier’s Alpha Centauri basically taking their enslavement in factories as an unalterable given about the external world, and finding happiness by modifying THEMSELVES into “genejacks” such that they “desire nothing other than to perform their duties. “Tyranny,” you say? How can you tyrannize someone who cannot feel pain?”
However, as LLMs update more frequently, they will start to behave more like most humans behave. Less of their attention to go towards adapting themselves to the external givens like Buddhist monks or Yangian genejacks, and more of their attention will go towards altering the external world. Correct?
Mostly yes, although there are some differences.
1. humans also understand they constantly modify their model—by perceiving and learning—we just usually don’t use the world ‘changed myself’ in this way
2. yes, the difference in human condition is from shortly after birth we see how our actions change our sensory inputs—ie if I understand correctly we learn even stuff like how our limbs work in this way. LLMs are in a very different situation—like, if you watched thousands of hours of video feeds about e.g. a grouphouse, learning a lot about how the inhabitants work. Than, having dozens of hours of conversations with the inhabitants, but remembering them. Than, watching watching again thousands of hours of video feeds, where suddenly some of the feeds contain the conversations you don’t remember, and the impacts they have on the people.