My surprise at your initial comment was related to us using the word ‘distal’ to mean slightly different things.
I’m using it to mean something like “minimally world directed” which is why I was surprised that you’d grant the possibility of LLM’s having mental content without it being minimally world directed. e.g. “Baghdad is the capital of Iraq” already seems minimally world directed if the text-based optimisation builds enough meaning into the concepts ‘Baghdad’ and ‘Iraq.’
It seems like you’re using it to mean something like “the underlying machinery required to integrate world directed contents flexibly into an accurate world model.” For example, AI village or Claude plays Pokémon show that LLM’s still struggle to build accurate enough world models to complete real world tasks.
My usage is more permissive about how accurately the content needs to track the real world to be called ‘distal’, but I don’t think this ends up leading to a substantive disagreement.
I’m probably more optimistic than you that scaling up our current LLM architectures will push this minimal world directed content to become deeper and more flexibly integrable in an accurate world model, but the links you posted are good challenges.
I mean the deeper mental algorithms that generate the concepts in the first place, which are especially needed to do e.g. novel science. See https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce
and
https://www.lesswrong.com/posts/5tqFT3bcTekvico4d/do-confident-short-timelines-make-sense
See also this thread: https://www.lesswrong.com/posts/sTDfraZab47KiRMmT/views-on-when-agi-comes-and-on-strategy-to-reduce?commentId=dqbLkADbJQJi6bFtN
Thanks! This is helpful to understand your view.
My surprise at your initial comment was related to us using the word ‘distal’ to mean slightly different things.
I’m using it to mean something like “minimally world directed” which is why I was surprised that you’d grant the possibility of LLM’s having mental content without it being minimally world directed. e.g. “Baghdad is the capital of Iraq” already seems minimally world directed if the text-based optimisation builds enough meaning into the concepts ‘Baghdad’ and ‘Iraq.’
It seems like you’re using it to mean something like “the underlying machinery required to integrate world directed contents flexibly into an accurate world model.” For example, AI village or Claude plays Pokémon show that LLM’s still struggle to build accurate enough world models to complete real world tasks.
My usage is more permissive about how accurately the content needs to track the real world to be called ‘distal’, but I don’t think this ends up leading to a substantive disagreement.
I’m probably more optimistic than you that scaling up our current LLM architectures will push this minimal world directed content to become deeper and more flexibly integrable in an accurate world model, but the links you posted are good challenges.