I basically agree that LLMs don’t seem all that inherently dangerous and am somewhat confused about rationalists’ reaction to them. LLMs seem to have some inherent limitations.
That said, I could buy that they could become dangerous/accelerate timelines. To understand my concern, let’s consider a key distinction in general intelligence: horizontal generality vs vertical generality.
By horizontal generality, I mean the ability to contribute to many different tasks. LLMs supersede or augment search engines in being able to funnel information from many different places on the internet right to a person who needs it. Since the internet contains information about many different things, this is often useful.
By vertical generality, I mean the ability to efficiently complete tasks with minimal outside assistance. LLMs do poorly on this, as they lack agency, actuators, sensors and probably also various other things needed to be vertically general.
(You might think horizontal vs vertical generality is related to breadth vs depth of knowledge, but I don’t think it is. The key distinction is that breadth vs depth of knowledge concerns fields of information, whereas horizontal vs vertical generality concerns tasks. Inputs vs outputs. Some tasks may depend on multiple fields of knowledge, e.g. software development depends on programming capabilities and understanding user needs, which means that depth of knowledge doesn’t guarantee vertical generality. On the other hand, some fields of knowledge, e.g. math or conflict resolution, may give gains in multiple tasks, which means that horizontal generality doesn’t require breadth of knowledge.)
While we have had previous techniques like AlphaStar with powerful vertical generality, they required a lot of data from those domains they functioned in in order to be useful, and they do not readily generalize to other domains.
Meanwhile, LLMs have powerful horizontal generality, and so people are integrating them into all sorts of places. But I can’t help but wonder—I think the integration of LLMs in various places will develop their vertical generality, partly by giving them access to more data, and partly by incentivizing people to develop programmatic scaffolding which increases their vertical generality.
So LLMs getting integrated everywhere may incentivize removing their limitations and speeding up AGI development.
I basically agree that LLMs don’t seem all that inherently dangerous and am somewhat confused about rationalists’ reaction to them. LLMs seem to have some inherent limitations.
That said, I could buy that they could become dangerous/accelerate timelines. To understand my concern, let’s consider a key distinction in general intelligence: horizontal generality vs vertical generality.
By horizontal generality, I mean the ability to contribute to many different tasks. LLMs supersede or augment search engines in being able to funnel information from many different places on the internet right to a person who needs it. Since the internet contains information about many different things, this is often useful.
By vertical generality, I mean the ability to efficiently complete tasks with minimal outside assistance. LLMs do poorly on this, as they lack agency, actuators, sensors and probably also various other things needed to be vertically general.
(You might think horizontal vs vertical generality is related to breadth vs depth of knowledge, but I don’t think it is. The key distinction is that breadth vs depth of knowledge concerns fields of information, whereas horizontal vs vertical generality concerns tasks. Inputs vs outputs. Some tasks may depend on multiple fields of knowledge, e.g. software development depends on programming capabilities and understanding user needs, which means that depth of knowledge doesn’t guarantee vertical generality. On the other hand, some fields of knowledge, e.g. math or conflict resolution, may give gains in multiple tasks, which means that horizontal generality doesn’t require breadth of knowledge.)
While we have had previous techniques like AlphaStar with powerful vertical generality, they required a lot of data from those domains they functioned in in order to be useful, and they do not readily generalize to other domains.
Meanwhile, LLMs have powerful horizontal generality, and so people are integrating them into all sorts of places. But I can’t help but wonder—I think the integration of LLMs in various places will develop their vertical generality, partly by giving them access to more data, and partly by incentivizing people to develop programmatic scaffolding which increases their vertical generality.
So LLMs getting integrated everywhere may incentivize removing their limitations and speeding up AGI development.