The skill of conceptual thinking is fairly mysterious to me. I think I’m differentially good at it, based on feedback from colleagues, but that’s not super clear.
Conceptual thinking kind of feels like annealing from the inside. I start with a bunch of ideas (e.g. natural latents, mean field theory, neural tangent kernels/influence functions, midtraining, inductive bias) and just wiggle them around to see what fits, ironing out the local inconsistencies in my worldview. As I spend more time turning an idea around in my head, my brain builds a better set of hashmaps for the concepts involved, so I can “zoom out” further and further, keeping more of it in view at any one time. Then I can resolve inconsistencies at larger and larger scales. Eventually the whole thing is globally consistent, and the final shape of it is a new mental object I can work with.
The skill of conceptual thinking is fairly mysterious to me. I think I’m differentially good at it, based on feedback from colleagues, but that’s not super clear.
Conceptual thinking kind of feels like annealing from the inside. I start with a bunch of ideas (e.g. natural latents, mean field theory, neural tangent kernels/influence functions, midtraining, inductive bias) and just wiggle them around to see what fits, ironing out the local inconsistencies in my worldview. As I spend more time turning an idea around in my head, my brain builds a better set of hashmaps for the concepts involved, so I can “zoom out” further and further, keeping more of it in view at any one time. Then I can resolve inconsistencies at larger and larger scales. Eventually the whole thing is globally consistent, and the final shape of it is a new mental object I can work with.