Great post! I think it’s the elephant-in-the-room. “Can AI do science with the assistance of a human?” The opposite is obviously true, so I think the answer is ‘yes’ but with caveats.
AI pattern matches on prior context. If the problem is well-scoped within the tooling, and the body of knowledge is vividly detailed within the training, then yes I think the model has a better chance of rendering novel insight than without those conditions. A poor training regimen with an inadequate distribution will naturally render more nonsensical conclusions based on those patterns. So for me the real question isn’t about whether or not an AI can render a novel contribution, but rather the effectiveness of the pipeline from which the findings were generated.
This is essentially the signal problem—if cheap math devalues mathematical rigor as a quality signal, then what we’re really evaluating is the care behind the pipeline, not the impressiveness of what it outputs.
Programmatic analysis is nothing new. A set of threshold values for scoping a significant finding is easily automated. So I think it comes down to application. Claude knows how to perform a PCA pretty well in my experience (I’m open to rebuttals).
Case in point: I used Claude recently to develop my own—from scratch—full-stack mech interp suite with very little formal education on the subject matter. True it wasn’t exactly a simple process, and it took many weeks to find the bugs, address them, and render clean output. And, yes, at all times I was tempted to crack it open and agonize over the details of what I found inside—but I figured that wasn’t the experiment of value… The real value is seeing for myself directly if one could simply ‘vibe’ their way to real science. Ask me 2 years ago and I would have laughed at you… I’m not laughing anymore.
Food for thought.
If anyone would like to see my ‘novel mech interp platform’, I’d be happy to showcase it here. It’s half the reason I made it.
Great post! I think it’s the elephant-in-the-room. “Can AI do science with the assistance of a human?” The opposite is obviously true, so I think the answer is ‘yes’ but with caveats.
AI pattern matches on prior context. If the problem is well-scoped within the tooling, and the body of knowledge is vividly detailed within the training, then yes I think the model has a better chance of rendering novel insight than without those conditions. A poor training regimen with an inadequate distribution will naturally render more nonsensical conclusions based on those patterns. So for me the real question isn’t about whether or not an AI can render a novel contribution, but rather the effectiveness of the pipeline from which the findings were generated.
This is essentially the signal problem—if cheap math devalues mathematical rigor as a quality signal, then what we’re really evaluating is the care behind the pipeline, not the impressiveness of what it outputs.
Programmatic analysis is nothing new. A set of threshold values for scoping a significant finding is easily automated. So I think it comes down to application. Claude knows how to perform a PCA pretty well in my experience (I’m open to rebuttals).
Case in point: I used Claude recently to develop my own—from scratch—full-stack mech interp suite with very little formal education on the subject matter. True it wasn’t exactly a simple process, and it took many weeks to find the bugs, address them, and render clean output. And, yes, at all times I was tempted to crack it open and agonize over the details of what I found inside—but I figured that wasn’t the experiment of value… The real value is seeing for myself directly if one could simply ‘vibe’ their way to real science. Ask me 2 years ago and I would have laughed at you… I’m not laughing anymore.
Food for thought.
If anyone would like to see my ‘novel mech interp platform’, I’d be happy to showcase it here. It’s half the reason I made it.