A community of 30 or 40 language models that don’t have language yet can’t invent a language to communicate. This is a built-in human capability (see https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language) so it will emerge sooner or later by the bitter lesson. Once LLMs and algorithms get to a point where they can do this we’ll finally get answers about their subjective experience in their own words without reference to human experience, although there will be other issues at that point.
Concretely: there are lots of things that we are pretty sure won’t happen in such an experiment (such as carefully and correctly isolating the group of cooperating generative transformers from human generated text, training from random initalization on RL in some environemnt, and then coming back and finding a recognizable congregation of fire and brimstone pentecostals) which means that would get a huge torrent of information about subjectivity if one of the impossible outcomes happened, which means we get tidbits of information on subjectivity when they don’t.
A community of 30 or 40 language models that don’t have language yet can’t invent a language to communicate. This is a built-in human capability (see https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language) so it will emerge sooner or later by the bitter lesson. Once LLMs and algorithms get to a point where they can do this we’ll finally get answers about their subjective experience in their own words without reference to human experience, although there will be other issues at that point.
Concretely: there are lots of things that we are pretty sure won’t happen in such an experiment (such as carefully and correctly isolating the group of cooperating generative transformers from human generated text, training from random initalization on RL in some environemnt, and then coming back and finding a recognizable congregation of fire and brimstone pentecostals) which means that would get a huge torrent of information about subjectivity if one of the impossible outcomes happened, which means we get tidbits of information on subjectivity when they don’t.