Hard to nail down exactly what I mean here, but I see traces of an objective account of what it’s subjectively like to be a person in correlations between oral traditions of tribes that share human ancestry but not any cultural connection. This could be done better with ideas expressed in languages that emerged recently (with the prototypical cases being schools for the deaf or hereditarily deaf populations inventing sign languages) but I haven’t found any pre-translation oral traditions in these languages.
A community of 30 or 40 language models that don’t have language yet can’t invent a language to communicate. This is a built-in human capability (see https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language) so it will emerge sooner or later by the bitter lesson. Once LLMs and algorithms get to a point where they can do this we’ll finally get answers about their subjective experience in their own words without reference to human experience, although there will be other issues at that point.
Concretely: there are lots of things that we are pretty sure won’t happen in such an experiment (such as carefully and correctly isolating the group of cooperating generative transformers from human generated text, training from random initalization on RL in some environemnt, and then coming back and finding a recognizable congregation of fire and brimstone pentecostals) which means that would get a huge torrent of information about subjectivity if one of the impossible outcomes happened, which means we get tidbits of information on subjectivity when they don’t.
Hard to nail down exactly what I mean here, but I see traces of an objective account of what it’s subjectively like to be a person in correlations between oral traditions of tribes that share human ancestry but not any cultural connection. This could be done better with ideas expressed in languages that emerged recently (with the prototypical cases being schools for the deaf or hereditarily deaf populations inventing sign languages) but I haven’t found any pre-translation oral traditions in these languages.
A community of 30 or 40 language models that don’t have language yet can’t invent a language to communicate. This is a built-in human capability (see https://en.wikipedia.org/wiki/Nicaraguan_Sign_Language) so it will emerge sooner or later by the bitter lesson. Once LLMs and algorithms get to a point where they can do this we’ll finally get answers about their subjective experience in their own words without reference to human experience, although there will be other issues at that point.
Concretely: there are lots of things that we are pretty sure won’t happen in such an experiment (such as carefully and correctly isolating the group of cooperating generative transformers from human generated text, training from random initalization on RL in some environemnt, and then coming back and finding a recognizable congregation of fire and brimstone pentecostals) which means that would get a huge torrent of information about subjectivity if one of the impossible outcomes happened, which means we get tidbits of information on subjectivity when they don’t.