Well technically speaking AIs aren’t explicitlypre-trained to predict what characters would do either, characters are an emergent feature extracted from next token prediction.
There are definitely some differences(AIs get way more character pre-training data, there’s an explicit separation of pre- and post-training, humans have lifetime memory) but overall I think the “agentic part is a subset of a predictive model” thing is pretty plausible in both cases.
AIs aren’t explicitlypre-trained to predict what characters would do.
I don’t understand, if a novel is in the pre-training where ‘scott decides to fly the spaceship to the moon’ it literally is trained to predict the character?
It has a general objective of next-token prediction for which modeling characters is a useful strategy. IMO it’s plausible that the human brain is “trained” on prediction to a large extent, for which modeling characters is also a useful strategy.
Sure. By “not explicitly pre-trained” I just mean to say that there’s nothing ‘special’ about the characters from the training algorithm’s point of view, so in this respect they’re not so different from a hypothetical general predictive algorithm in humans(although actually I guess the human brain attaches special salience to other people, but regardless...)
Well technically speaking AIs aren’t explicitly pre-trained to predict what characters would do either, characters are an emergent feature extracted from next token prediction.
There are definitely some differences(AIs get way more character pre-training data, there’s an explicit separation of pre- and post-training, humans have lifetime memory) but overall I think the “agentic part is a subset of a predictive model” thing is pretty plausible in both cases.
I don’t understand, if a novel is in the pre-training where ‘scott decides to fly the spaceship to the moon’ it literally is trained to predict the character?
It has a general objective of next-token prediction for which modeling characters is a useful strategy. IMO it’s plausible that the human brain is “trained” on prediction to a large extent, for which modeling characters is also a useful strategy.
Maybe you meant to say they aren’t pre-trained to *model* characters? I still think they are literally trained to predict?
Sure. By “not explicitly pre-trained” I just mean to say that there’s nothing ‘special’ about the characters from the training algorithm’s point of view, so in this respect they’re not so different from a hypothetical general predictive algorithm in humans(although actually I guess the human brain attaches special salience to other people, but regardless...)