“So we would need to find a hypothesis where we accidentally already made all the necessary experiments and even described the intermediate findings (because LLMs are good at words, but probably suck at analyzing the primary data), but we somehow failed to connect the dots. Not impossible, but requires a lot of luck.”
Exactly: untested hypotheses that LLMs already have enough data to test. I wonder how rare such hypotheses are.
It strikes me as wild that LLMs have ingested enormous swathes of the internet, across thousands of domains, and haven’t yet produced genius connections between those domains (eg between psychoanalysis and tree root growth). Cross Domain Analogies seem like just one example of ripe category of hypotheses that could be tested with existing LLM knowledge.
“So we would need to find a hypothesis where we accidentally already made all the necessary experiments and even described the intermediate findings (because LLMs are good at words, but probably suck at analyzing the primary data), but we somehow failed to connect the dots. Not impossible, but requires a lot of luck.”
Exactly: untested hypotheses that LLMs already have enough data to test. I wonder how rare such hypotheses are.
It strikes me as wild that LLMs have ingested enormous swathes of the internet, across thousands of domains, and haven’t yet produced genius connections between those domains (eg between psychoanalysis and tree root growth). Cross Domain Analogies seem like just one example of ripe category of hypotheses that could be tested with existing LLM knowledge.