This seems to be a consequence of having a large but not-actually-that-deep-in-serial-steps net trained on next token prediction of a big pile of human data. AI doesn’t have to be like that—I expect something that can competently choose which cognitive strategies to execute will be much better at multiplication than a human, but it’s hard to get to that kind of AI by predictive training on a big pile of human data.
I think this is the point. Existing training creates something like System 1, which now happens to match what humans find “natural”. Something else is probably needed to make math “natural” for ML models.
This seems to be a consequence of having a large but not-actually-that-deep-in-serial-steps net trained on next token prediction of a big pile of human data. AI doesn’t have to be like that—I expect something that can competently choose which cognitive strategies to execute will be much better at multiplication than a human, but it’s hard to get to that kind of AI by predictive training on a big pile of human data.
I think this is the point. Existing training creates something like System 1, which now happens to match what humans find “natural”. Something else is probably needed to make math “natural” for ML models.