Both. My understanding of the difficulty for clinical trials is based primarily on the following 2017 SSC post, which is well worth (re)reading in full:My IRB Nightmare. To my knowledge, the current SoTA for brain modelling is TRIBE v2, which is trained on 1,000 hours of fMRI across 720 subjects. The largest dataset of neuro-language data I am aware of is 10k hours long. Another risk is that in silico simulations would as a substrate be closer to hosting moral patients than LLMs, and it would be challenging to provably verify that they are being handled responsibly and with ethical treatment.
Which part are you thinking is hard? The in silico simulations, or large-scale recruitment?
Both. My understanding of the difficulty for clinical trials is based primarily on the following 2017 SSC post, which is well worth (re)reading in full: My IRB Nightmare. To my knowledge, the current SoTA for brain modelling is TRIBE v2, which is trained on 1,000 hours of fMRI across 720 subjects. The largest dataset of neuro-language data I am aware of is 10k hours long. Another risk is that in silico simulations would as a substrate be closer to hosting moral patients than LLMs, and it would be challenging to provably verify that they are being handled responsibly and with ethical treatment.
Curious if this could be a pitch for https://simile.ai/blog/the-simulation-company to take on this challenge / maybe they would be a well situated company to develop such an eval.