Yeah, I’ve seen those! They do express similar ideas, and I think Chris does serious work. I completely agree with the claim “the parameter-function map is strongly biased towards simple functions,” basically for SLT reasons (though one has to be careful here to avoid saying something tautological, it’s not as simple as “SLT says learning is biased towards lower LLC solutions, therefore it has a simplicity bias”).
There’s a good blog post discussing this work that I read a few years ago. Keeping in mind I read this three years ago and so this might be unfair, I remember my opinion on that specific paper being something along the lines of “the ideas are great, the empirical evidence seems okay, and I don’t feel great about the (interpretation of the) theory.” In particular I remember reading this comment thread and agreeing with the perspective of interstice, for whatever that’s worth. But I could change my mind.
Yeah, I’ve seen those! They do express similar ideas, and I think Chris does serious work. I completely agree with the claim “the parameter-function map is strongly biased towards simple functions,” basically for SLT reasons (though one has to be careful here to avoid saying something tautological, it’s not as simple as “SLT says learning is biased towards lower LLC solutions, therefore it has a simplicity bias”).
There’s a good blog post discussing this work that I read a few years ago. Keeping in mind I read this three years ago and so this might be unfair, I remember my opinion on that specific paper being something along the lines of “the ideas are great, the empirical evidence seems okay, and I don’t feel great about the (interpretation of the) theory.” In particular I remember reading this comment thread and agreeing with the perspective of interstice, for whatever that’s worth. But I could change my mind.