I think we share a lot of perspective, so I’m focusing on the bit that stands out to me as needing more justification.
a lot of claims of ‘you can only know [X] by running a physical experiment’ are very wrong, again a Skill Issue
What do you mean here? Are you pointing to in silico simulation, inference etc.? Which claims are ‘a lot of claims’? Depending on how this is qualified, I could imagine agreeing strongly or disagreeing vociferously. I wrote You Can’t Skip Exploration on this topic. Possibly the main points relevant here are:
in silico is only possible when you’ve previously built sufficiently detailed models, on the basis of (often very much) physical experiment. Sim2real bites regardless, especially if you’re simulating novel interactions at the boundary of the known.
getting research taste to drive good (high VOI) experiment choice is nothing more than getting somewhat generalising heuristic VOI estimator/generators, which you get only by observing (perhaps indirectly) sufficiently similar experiments (and their eventual VOI). You can reason on top of that for (I think very diminishing) marginal gains. This makes sample efficiency (at gathering research taste), and experiment throughput critical.
You can get some sort of bootstrap to research taste ‘stock’ by slurping up all the textbooks and by interviewing existing experts and such. Unclear how much, maybe lots (but it’ll depreciate when the frontier of the known moves forward).
I think we share a lot of perspective, so I’m focusing on the bit that stands out to me as needing more justification.
What do you mean here? Are you pointing to in silico simulation, inference etc.? Which claims are ‘a lot of claims’? Depending on how this is qualified, I could imagine agreeing strongly or disagreeing vociferously. I wrote You Can’t Skip Exploration on this topic. Possibly the main points relevant here are:
in silico is only possible when you’ve previously built sufficiently detailed models, on the basis of (often very much) physical experiment. Sim2real bites regardless, especially if you’re simulating novel interactions at the boundary of the known.
getting research taste to drive good (high VOI) experiment choice is nothing more than getting somewhat generalising heuristic VOI estimator/generators, which you get only by observing (perhaps indirectly) sufficiently similar experiments (and their eventual VOI). You can reason on top of that for (I think very diminishing) marginal gains. This makes sample efficiency (at gathering research taste), and experiment throughput critical.
You can get some sort of bootstrap to research taste ‘stock’ by slurping up all the textbooks and by interviewing existing experts and such. Unclear how much, maybe lots (but it’ll depreciate when the frontier of the known moves forward).