I dunno how to respond to this question. It reads like it wants to make a lot of weird-to-me assumptions about the relationship between human cognition and currently popular ML methods. Like, I could give an object-level answer but that feels inadequate considering.
I think what struck me most was the assumption the success of the next token prediction objective in today’s ML implies something specific about how human cognition works (especially to the point where we might hypothesize that most of it is just generic prediction + training data).
If you look at the kinds of cognitive architectures that shoot for a brain-like structure, like ACT-R or Leabra or Yann LeCun’s thing or SPAUN, most of the components are doing things that are not very similar to GPT-style next token prediction.
Hmm, interesting. I wonder if this is an example of Carcinisation, where you can get some ways toward imitating/implementing cognition from multiple directions.
I dunno how to respond to this question. It reads like it wants to make a lot of weird-to-me assumptions about the relationship between human cognition and currently popular ML methods. Like, I could give an object-level answer but that feels inadequate considering.
hmm, which part is weird?
I think what struck me most was the assumption the success of the next token prediction objective in today’s ML implies something specific about how human cognition works (especially to the point where we might hypothesize that most of it is just generic prediction + training data).
If you look at the kinds of cognitive architectures that shoot for a brain-like structure, like ACT-R or Leabra or Yann LeCun’s thing or SPAUN, most of the components are doing things that are not very similar to GPT-style next token prediction.
Hmm, interesting. I wonder if this is an example of Carcinisation, where you can get some ways toward imitating/implementing cognition from multiple directions.