The thing I am trying to point at is what happens in the CoT-reasoning. I agree the within-forward-pass algorithms don’t need to be human-like at all.
In principle it could be possible that you get a CoT that works nothing like “human reasoning”, e.g. because there is some structure in some codebases or in some procedurally generated reports common in pretraining that are useful for reasoning, but I am not aware of such examples and on priors that seems not that likely because that next was not “made to be useful reasoning” (while human-generated reasoning is).
The thing I am trying to point at is what happens in the CoT-reasoning. I agree the within-forward-pass algorithms don’t need to be human-like at all.
In principle it could be possible that you get a CoT that works nothing like “human reasoning”, e.g. because there is some structure in some codebases or in some procedurally generated reports common in pretraining that are useful for reasoning, but I am not aware of such examples and on priors that seems not that likely because that next was not “made to be useful reasoning” (while human-generated reasoning is).